Why you shouldn't use singleton DataContexts in Entity Framework

I have been struggling recently with putting an end to a problem caused my optimistically keeping the same DataContext alive in a singleton pattern. Let me tell you what I found and why I think it is bad.

Getting it wrong will cripple your site

The first and foremost reason why you shouldn't use it is that if you set it up wrong the repercussions are critical. Here's my horror story: after ages of head scratching, wondering why the CPUs are continuously nearing 100% and memory leaks are ubiquitous I finally figured out that DataContexts were not being disposed of. Of course this is by design - lovely singleton pattern...

Anyway, not only were they not being disposed of they were being instantiated with every access of the data. So we ended up with piles of open connections, which I assume were being terminated randomly by some critical process. This causes huge memory utilisation and huge CPU usage trying to reallocate memory.

Fine so it was implemented incorrectly, but it is practically impossible to see that this is happening during development without shelling out for a decent entity framework profiler (like Entity Profiler - it is very much worth it if you don't trust EF).

Data isn't in sync sometimes...

When we were using the singleton well the data often seemed out of sync with the database. It's as if the ObjectContext knew best and didn't really care what the database had to say. This is not wise when you are working on a web farm since your different servers will have a different view of what the data is in its memory.

The way to resolve this is to begin a new connection... In comes the problem of multiplying open and undisposed connections.

Where's the performance gain anyway?

Has anyone any idea of how expensive it is to open a DataContext connection? Answer: "A DataContext is lightweight and is not expensive to create" [citation]. So why is everyone so intent on saving the hassle of creating a DataContext?

You are probably saving a few 10s of milliseconds. The word micro optimisation springs to mind - in which case you probably shouldn't be using Entity Framework.

It allows poor coding

Having a DataContext that is never closed allows you to lazy load whenever you want. You may have left your Service and now be in your Controller or worse still, your View. Accessing the database from a View is asking for performance problems as I'm sure you didn't allow that intentionally. It is most likely because you forgot to eager load all the data you need to populate your view.

This also makes it hard to keep things nice and streamlined since you don't know where all your queries are coming from.

Conclusion

If you are a guru of Entity Framework and you think a Singleton DataContext is for you then by all means go for it. If you aren't convinced you need it and you're just experimenting you could encounter unexpected and serious problems. Don't use singleton because you think it sounds cool!

Also, during fixing up my problems I found the Entity Profiler invaluable. It tracks how many connections have been opened and closed (hopefully they will be equal). It also shows all the actual SQL that is being produced, the results, the query plan and even tips on how to sort your query out. Invaluable!

Visual Studio's load test and 400 Bad request errors

When trying some load testing with Visual Studio Ultimate edition. My load test received lots of 400 bad request errors. But why?

I had a look at my page that was claiming to be full of errors and it was fine. Even Fiddler reported that there were no 400's being returned from my page.

I took a look more closely at the URL that was returning the 400 and it was not the same as the one I put into the Web Performance test. It had suffixed my URL with %3C%25 - I had a look and this is URL encoding for <%. The 'start writing code' symbol in web forms pages/views.

Workaround to fix this 400 Bad Request %3C%25 problem

I routed about my View and found it could be a problem with the ad calls in the page. They make https ad calls so maybe this is the problem since I heard something about https not being supported properly in load test.

So I stripped the ad call parts out of my View and everything is working fine now.

Obviously this will now affect my page speed so not very accurate. Not impresses given the cost of Visual Studio Ultimate

Multiple generic constraints in C#

I had some trouble using my nice new generic class in C#. I want to ensure one of the types passed in inherited from a particular class. This class can be instantiated using the new key word however the compile time is not aware of this

I found, using this unnatural syntax, it is possible to constrain the type parameters in multiple ways. Like so:

public class MyClass <T, U> : MyBaseClass <T> where T : class where U : AnotherClass, new()
{
    public AnotherClass AnotherClass { get; set; }
    public MyClass()
    {
        AnotherClass = new U();
    }
    //my class
}

This works fine and can be very useful at times

Google's DFP adservices have no secure HTTPS ability!

I have been getting security warning on my secure pages because an unsecure resource has crept in since I've started using Google Ad Services.

You will struggle using Google's "DFP" adservices on secure https pages since Google is not hosting their http://partner.googleadservices.com/gampad/google_service.js file on anything capable of serving securely.

Well, they will work, but they will undermine your attempts to reassure users they are on a secure page. In IE8 for example you get the huge AAAAAAAAAGGGGHHHH!!!! IT'S UNSECURE THEREFORE YOU ARE DEFINITELY BEING HACKED. YOU ACTUALLY NEED TO CLICK 'NO' TO PROCEED UNTOUCHED!!! style ridiculous error.

So what the hell are Google ad services playing at?

Usually, you would just reference the secure version of jQuery, skimlinks, OpenX etc so avoid this problem. As a someone trying to push from OpenX to DFP I am disheartened. Will they ever support https? "Not at this time."

Google: sometimes you can be quite disappointing. I first found this problem October 2010. Still there now... (23^rd Feb 2011)

Update - It now works!

It now works. Thanks to a commenter I thought I'd recheck the situation and it looks like Google have resolved this issue as of 7^th July 2011 (or some point before). You can access https://partner.googleadservices.com/gampad/google_service.js which will now keep your secure pages happy.

SEO Redirecting URLs to lowercase excluding static files

As you may know Google treats different cased URLs as different URL entirely. If they find both cases they will judge this as duplicate content so it is a good idea to redirect requests for a URL containing uppercase to lowercase.

This is made easy with the URL Rewriting Module 2.0. You can use the "Enforce lowercase URLs" template to do most of the work for you.

This will create a pattern match of [A-Z] (as in if there is even one capital in the URL) it will 301 redirect to {ToLower:{URL}} which is fairly obvious what it does.

What about static resources files?

You don't really want these redirects to happen since you don't really want search engines to index them so it takes up needless resource. Also, for my particular set up it fails to send them (some sort of weird, secure + load balanced + redirecting issue).

1. Open the Conditions section of your lowercase redirect rule and click Add...
2. Set Condition input to {URL} (which doesn't look at any querystrings).
3. Set Check if input string to Does Not Match the Pattern
4. Change the Pattern field to one that suits you. I have used ^.+\.(axd|gif|jpg|jpeg|png|css|js|svc|ashx)$ since I do not what files with these extension redirected. You may have more. Or less
5. Ignore case should be checked
6. Click OK and Apply your new rule

Your site should now send www.mysite.com/Home/Index to www.mysite.com/home/index with a permanent 301 redirect and leave www.mysite.com/WebResource.axd alone.

Multiple duplicate cookies and one blank cookie

I noticed we had a problem where there were duplicate cookies on my site. One was correct but the other was carrying an empty string.

It took a lot of debugging and use of Fiddler but I found out something very odd with the .NET framework that was causing this problem.

The oddity

It would seem that evaluating the response's cookies creates one if it doesn't already exist. Here's an example:

if (Response.Cookies["myCookie"] == null)
{
    var cookie = new HttpCookie("myCookie", "myValue");
    cookie.Domain = ".britishdeveloper.co.uk";
    cookie.Expires = DateTime.Now.AddDays(7);
    Response.Cookies.Add(cookie);
}
//relax because you now have a cookie with the intended domain and expiry. Right?

Line 1 actually creates the myCookie cookie! If you ever evaluate the property indexer it will create that cookie with no value, no expiration, no domain. Nothing, but it will still exist! Fiddler will show you this in your Response headers: Set-Cookie: myCookie. Also, line 1 will always evaluate to false because of this.

How to avoid the blank cookie

If you are checking for its existence you should NOT use the index property as it will make a new cookie if it doesn't exist.

Instead, use this code that star developer Leo discovered to resolve the issue:

if (!Response.Cookies.AllKeys.Contains("myCookie"))
{
    var cookie = new HttpCookie("myCookie", "myValue");
    cookie.Domain = ".britishdeveloper.co.uk";
    cookie.Expires = DateTime.Now.AddDays(7);
    Response.Cookies.Add(cookie);
}
//relax because you now have a cookie with the intended domain and expiry. Right? YES!

Line 1 now evaluates if myCookie exists in the collection of all response cookies rather than accessing (and creating) it directly. So now you know, hopefully it won’t cause you the hours of debugging it cost myself and Leo.

Note: This is not true of Request.Cookies - only Response.Cookies. Lovely bit of inconsistency there

Proxy server connection failed

I got a mysterious "Unable to connect to the proxy server" when trying to connect to any website. The other useless message was "Error 130 (net::ERR_PROXY_CONNECTION_FAILED): Proxy server connection failed."

It turns out I was using a proxy. I did have fiddler running earlier and I shut it down by ending its process via Task Manager.

Looks like Fiddler needs you to shut it down properly when you are done with it because it obviously needs to stop being a proxy to your internet connection.

Solution

Simply open Fiddler up again and then close it properly and it should close it's current proxy service and you should be fine.

OutOfMemoryException when cropping image

When cropping an image using System.Drawing.Bitmap.Clone(Rectangle rect, PixelFormat format) I was receiving an OutOfMemoryException complaining that I was "Out of memory" when trying to crop an Image bitmap in C#.

So I started to assume that I was out of memory as in physical storage on my machine as you might expect. Turns out that that is not in fact the case as you would surely assume...

What's wrong with this clone?!

Well it turns out that the cropping was happening on an image (1 pixel) smaller than the desired cropping size. So I changed my logic about a bit to essentially make sure that the source image is at least the size of the desired crop size. And now the OutOfMemoryException on my Image processing code is working just fine.

Regex [x-y] range in reverse order

I started receiving a 'System.ArgumentException' error with a decription of 'parsing "[A-Za-z0-9_- %]+\.(bmp|jpg|jpeg|gif|png)" - [x-y] range in reverse order' after I wrote a C# Regex in my ASP.NET MVC web application.

I was using a RegularExpression Validator in my ASP.NET MVC app but it wasn't having any of it. I was looking at "A-Z" etc thinking IT IS THE RIGHT ORDER!

Turns out it is because I have a hyphen in another part of my regex expression. The one after the underscore.

Solution

Turns out hyphens have to appear at either the front or the end of the regular expression - or be escaped with a \. So now I have updated my regular expression from
[RegularExpression(@"[A-Za-z0-9_- %]+\.(bmp|jpg|jpeg|gif|png)")]
to
[RegularExpression(@"[A-Za-z0-9_ %-]+\.(bmp|jpg|jpeg|gif|png)")]

Working solution breaks on new machine. IIS setup with URL Rewrite Module

A solution that works fine on one machine will not work on a different development machine. I just received a 503 service unavailable error and the IIS rewrite module was giving an error saying "There was an error while performing this operation" and "Unrecognized attribute 'trackAllCaptures'"

When I get a massive IIS style, 'light blue screen of death' error message it is usually something to do with web.config not recognising some attribute. Clicking about the various modules in my site through IIS I soon discovered that it was the URL rewrite module specifically I was having problems with. It was saying that trackAllCaptures was an unrecognised attribute.

It was complaining about not recognising a particular node. This was because I hadn't got IIS URL Rewriter installed on my new machine. Installing it works a treat!