ASP.NET WebApi – Breaking change to MediaTypeFormatters

In my preview post, I described an implementation that would allow multipart/form-data image uploads to be binded to an ImageMedia model by implementing the MediaTypeFormatter. That post was based on the beta api and has since been changed a few times.

In the previous post, the method signature for ReadFromStreamAsync was:

        protected async override Task OnReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            FormatterContext formatterContext)

This allowed us to get at the request via the FormatterContext like this:

            HttpRequestMessage request = formatterContext.Request;

A change was then made that meant we could no longer access the HttpRequestMessage, and therefore no longer access the HttpContent property of the request.

Essentially, this change removed the FormatterContext parameter, and in its place passed an IFormatterLogger:

        public async override Task ReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            IFormatterLogger formatterLogger)

On the 15 August 2012, the ASP.NET WebApi was Released , and with it came a welcome change.

The team decided to replace the HttpContentHeaders parameter with the HttpContent allowing access to all the extension methods that come with it. The implementation I provided, depended on ReadAsMultipartAsync.

This method itself has also changed, as it no longer returns a collection of HttpContent, and now returns MultipartMemoryStreamProvider . The provider then allows access to the HttpContent collection via the Contents parameter.

Phew! So after all that, we can now have our ImageMediaTypeFormatter extract our image stream once again.

TL;DR
Enough of the pleasantries, here’s the updated ReadFromStreamAsync implementation.


        public async override Task<object> ReadFromStreamAsync(
                Type type, Stream stream, HttpContent content,
                IFormatterLogger formatterLogger)
        {
            if (!content.IsMimeMultipartContent())
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var provider = await content.ReadAsMultipartAsync();

            var mediacontent = provider.Contents.First(x =>
                SupportedMediaTypes.Contains(x.Headers.ContentType));

            string fileName = mediacontent.Headers.ContentDisposition.FileName;
            string mediaType = mediacontent.Headers.ContentType.MediaType;

            using (var imgstream = await mediacontent.ReadAsStreamAsync())
            {
                byte[] imagebuffer = imgstream.ReadFully();
                return new ImageMedia(fileName, mediaType, imagebuffer);
            }
        }

ASP.NET WebApi upload image with custom MediaTypeFormatter

UPDATE: changes to WebApi mean this code no longer works, see here for update.

I’ve been reading a few tutorials on www.asp.net and come across a good article demonstrating the basics of MediaTypeFormatters.

I’m currently working on a personal project utilising WebApi to post an image for OCR. I don’t need to save this file, I just need a byte array of it to perform some character recognition using Tesseract. One of the tutorials on http://www.asp.net, Forms and Multi-part MIME, describes how to upload an image by reading the multipart data directly inside the controller, and saving it to appdata folder.

This doesn’t seem ideal for my scenario, firstly, I don’t need to store the images and, more importantly, it makes it difficult to test.

Ideally, I needed my actions signature to be like this:

[HttpPost]
public Task<string> Upload(ImageMedia media)

Where ImageMedia looks like this:

    public class ImageMedia
    {
        public string FileName { get; set; }
        public string MediaType { get; set; }
        public byte[] Buffer { get; set; }

        public ImageMedia(string fileName, string mediaType, byte[] imagebuffer)
        {
            FileName = fileName;
            MediaType = mediaType;
            Buffer = imagebuffer;
        }
    }

Now, implementing a new MediaTypeFormatter I can extract the image and populate my ImageMedia model.

Firstly, we need to to tell the framework what media types the MediaFormatter applies to, we can do this by adding MediaTypeHeaderValues to the list of SupportedMediaTypes inside the constructor, something like this does the job:

        public ImageMediaFormatter()
        {
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpeg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/png"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("multipart/form-data"));
        }

We also need to specifiy what model types the formatter can write/read, because I’m only reading from the stream to my ImageMedia, I’ll need to override CanReadType comparing the Type passed in to typeof(ImageMedia):

        protected override bool CanReadType(Type type)
        {
            return type == typeof(ImageMedia);
        }

And I’ll simply return false from CanWriteType:

        protected override bool CanWriteType(Type type)
        {
            return false;
        }

Now for the actual work:

1. First check that the request is IsMimeMultipartContent
2. Then read the content of the request as Multipart.
3. We the find the part that that matches one of our SupportedMediaTypes, this will be the image we’re after.
4. We extract te fileName and mediaType from the headers of the content we retrieved.
5. then on content call ReadAsStreamAsync()
6. as ImageMedia wants a byte[] I call a handy helper “ReadFully” which is an extension method on Stream, (written by Jon Skeet)

       protected async override Task OnReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            FormatterContext formatterContext)
        {
            HttpRequestMessage request = formatterContext.Request;
            if (!request.Content.IsMimeMultipartContent())
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var parts = await request.Content.ReadAsMultipartAsync();
            var content = parts.First(x =>
                SupportedMediaTypes.Contains(x.Headers.ContentType));

            string fileName = content.Headers.ContentDisposition.FileName;
            string mediaType = content.Headers.ContentType.MediaType;

            using (var imgstream = await content.ReadAsStreamAsync())
            {
                byte[] imagebuffer = imgstream.ReadFully();
                return new ImageMedia(fileName, mediaType, imagebuffer);
            }
        }

And Jon Skeets read fully:

        public static byte[] ReadFully(this Stream input)
        {
            byte[] buffer = new byte[16 * 1024];
            using (MemoryStream ms = new MemoryStream())
            {
                int read;
                while ((read = input.Read(buffer, 0, buffer.Length)) &gt; 0)
                {
                    ms.Write(buffer, 0, read);
                }
                return ms.ToArray();
            }
        }

And thats it!!! So now the controller is no longer responsible for extracting the content of the request, and it can be left with simply coordinating it. Our ImageMediaFormatter will kick in any time an action on our WebApi requires an ImageMedia passed in.

For clarity, here’s the ImageMediaFormatter in full:


 public class ImageMediaFormatter : MediaTypeFormatter
    {
        public ImageMediaFormatter()
        {
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpeg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/png"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("multipart/form-data"));
        }

        protected override bool CanReadType(Type type)
        {
            return type == typeof(ImageMedia);
        }

        protected override bool CanWriteType(Type type)
        {
            return false;
        }

        protected async override Task OnReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            FormatterContext formatterContext)
        {
            HttpRequestMessage request = formatterContext.Request;
            if (!request.Content.IsMimeMultipartContent())
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var parts = await request.Content.ReadAsMultipartAsync();
            var content = parts.First(x =>
                SupportedMediaTypes.Contains(x.Headers.ContentType));

            string fileName = content.Headers.ContentDisposition.FileName;
            string mediaType = content.Headers.ContentType.MediaType;

            using (var imgstream = await content.ReadAsStreamAsync())
            {
                byte[] imagebuffer = imgstream.ReadFully();
                return new ImageMedia(fileName, mediaType, imagebuffer);
            }
        }
    }

Integration Tests – Tests Fail When Network Unavailable

I code a lot whilst travelling to work on the train. One thing that has bugged me is integration tests that rely on remote services will fail unless I’ve tethered my phone. Having those few red tests littering my test results kind of irks me and leads me to run the tests less often or not at all.

What I wanted was an ignore attribute that could detect if the network was unavailable and ignore the test, but I couldn’t find anythign to suit. As there isn’t great support for extending test attributes, I decided to build support in to my IntergrationBaseTest class.

This could have been solved easily by placing these two lines at the start of my test:

            
           if(!NetworkInterface.GetIsNetworkAvailable())
                Assert.Inconclusive();

The above is simple but I wanted to explore the idea of custom test attributes to see what I could come up with.

What first sprang to mind was ASP.NET MVC’s ActionFilterAttributes. MSDN describes ActionFilterAttributes as the following:

ActionFilterAttribute is an abstract class that has four virtual methods that you can override: OnActionExecuting, OnActionExecuted, OnResultExecuting, and OnResultExecuted.

The ASP.NET MVC framework will call the OnActionExecuting method of your action filter before it calls any action method that is marked with your action filter attribute. Similarly, the framework will call the OnActionExecuted method after the action method has finished.

Taking this approach and applying it to MSTest I created the below abstract attribute:

  
    public abstract class CustomTestAttribute : Attribute
    {
        public virtual void OnTestExecuting(TestContext context) {}
        public virtual void OnTestExecuted(TestContext context) {}
    }

I marked the methods as virtual so as not to force derived classes to implement both. As you might have guessed the OnTestExecuting maps to methods marked with the TestInitializeAttribute and the OnTestExecuted maps to the TestCleanupAttribute in MSTest. The benefit we gain here is the ability to descriminate down to the method level within a class.

To support the new attribute the base class is going to have to know about it and execute the respective methods in the corresponding initialize and cleanup methods.

First, to deal with multiple attributes as a single unit, I created a CompositeTestAttribute, deriving form the CustomTestAttribute:

  
    public sealed class CompositeTestAttribute : CustomTestAttribute
    {
        private readonly IEnumerable<CustomTestAttribute> attributes;

        public CompositeTestAttribute(IEnumerable<CustomTestAttribute> attributes)
        {
            this.attributes = attributes;
        }

        public override void OnTestExecuting(TestContext context)
        {
            foreach (var customTestAttribute in attributes)
            {
                customTestAttribute.OnTestExecuting(context);
            }
        }

        public override void OnTestExecuted(TestContext context)
        {
            foreach (var customTestAttribute in attributes)
            {
                customTestAttribute.OnTestExecuting(context);
            }
        }
    }

To determine which attributes the executing test is decorated with we can use the TestContext which gets injected by mstest at runtime, all you need to do is dangle a property on the base class:

  
	public TestContext TestContext { get; set; }

Using the TestContext we can access the “FullyQualifiedTestClassName” and “TestName” properties. This gives us enough information to generate a type and access the attributes:

  
        private IEnumerable<CustomTestAttribute> GetCustomTestAttributes()
        {
            IEnumerable<CustomTestAttribute> attributes = Enumerable.Empty<CustomTestAttribute>();
            string className = TestContext.FullyQualifiedTestClassName;
            string methodName = TestContext.TestName;

            Type testClassType = Type.GetType(className);
            if (testClassType != null)
            {
                MemberInfo mi = testClassType.GetMethod(methodName);
                attributes = mi.GetCustomAttributes<CustomTestAttribute>();
            }

            return attributes;
        }

All that’s needed now is to invoke OnTestExecuting and OnTestExecuted in the TestInitialize and TestCleanup methods respectively.

  
        [TestInitialize()]
        public void MyTestInitialize()
        {
            new CompositeTestAttribute(GetCustomTestAttributes())
                .OnTestExecuting(this.TestContext);
        }

        [TestCleanup()]
        public void MyTestCleanup()
        {
            new CompositeTestAttribute(GetCustomTestAttributes())
                .OnTestExecuted(this.TestContext);
        }

Now, that the infrastucture is there, we can start implementing CustomTestAttributes. Which brings me back to the reason for all this.
To determine if the Network is available, there is a handy helper method located in “System.Net.NetworkInformation.NetworkInterface” called

“GetIsNetworkAvailable()”

  
    public class IgnoreWhenNetworkUnavailableAttribute : CustomTestAttribute
    {
        public override void OnTestExecuting(TestContext context)
        {
            if(!NetworkInterface.GetIsNetworkAvailable())
                Assert.Inconclusive();
        }
    }

Now to make use of this, all we have todo is derive from the IntegrationBaseTest and decorate our network dependent test methods. 🙂

For clarity, below is the full code. Although this could have been achieved by far simpler means, it was an interesting exercise, that allowed me to delve into some code and have a little fun. I hope you find this helpful and put CustomTestAttributes to use in scenarios I haven’t thought of.

  
 /// <summary>
    /// Summary description for IntegrationTestBase
    /// </summary>
    [TestClass]
    public class IntegrationTestBase
    {
	public TestContext TestContext { get; set; }

        [TestInitialize()]
        public void MyTestInitialize()
        {
            new CompositeTestAttribute(GetCustomTestAttributes())
                .OnTestExecuting(this.TestContext);
        }

        [TestCleanup()]
        public void MyTestCleanup()
        {
            new CompositeTestAttribute(GetCustomTestAttributes())
                .OnTestExecuted(this.TestContext);
        }

        private IEnumerable<CustomTestAttribute> GetCustomTestAttributes()
        {
            IEnumerable<CustomTestAttribute> attributes = Enumerable.Empty<CustomTestAttribute>();
            string className = TestContext.FullyQualifiedTestClassName;
            string methodName = TestContext.TestName;

            Type testClassType = Type.GetType(className);
            if (testClassType != null)
            {
                MemberInfo mi = testClassType.GetMethod(methodName);
                attributes = mi.GetCustomAttributes<CustomTestAttribute>();
            }

            return attributes;
        }
    }

    public class IgnoreWhenNetworkUnavailableAttribute : CustomTestAttribute
    {
        public override void OnTestExecuting(TestContext context)
        {
            if(!NetworkInterface.GetIsNetworkAvailable())
                Assert.Inconclusive();
        }
    }

    public abstract class CustomTestAttribute : Attribute
    {
        public virtual void OnTestExecuting(TestContext context) {}
        public virtual void OnTestExecuted(TestContext context) {}
    }

    public sealed class CompositeTestAttribute : CustomTestAttribute
    {
        private readonly IEnumerable<CustomTestAttribute> attributes;

        public CompositeTestAttribute(IEnumerable<CustomTestAttribute> attributes)
        {
            this.attributes = attributes;
        }

        public override void OnTestExecuting(TestContext context)
        {
            foreach (var customTestAttribute in attributes)
            {
                customTestAttribute.OnTestExecuting(context);
            }
        }

        public override void OnTestExecuted(TestContext context)
        {
            foreach (var customTestAttribute in attributes)
            {
                customTestAttribute.OnTestExecuting(context);
            }
        }
    }

Extracting the Func

Recently, I’ve found myself using Generic Delegates quite a lot, and in particular the Func<> delegate. Delegates can be a great way of reducing duplication and increasing code re-use.

The following is a rather contrived example, and mostly useless, but it helps to illustrate the process of what I like to call “Extracting the Func”

Say you have a method that writes to the console, the result of adding two numbers together:


public void WriteResultOfCalculation(int x, int y)
{
     Console.Write((x + y).ToString());
}

But then, u decide you want similar functionality, but instead of adding x and y you want to multiply x and y (besides obvious ways of doing this) you could extract the calculation into a Func delegate like this:


public void WriteResultOf(Func<int, int, string> calculation)
{
     Console.Write(calculation);
}

WriteResultOf((x,y) => (1 * 2));

Here we have pushed the responsibility of performing the calculation to the consumer. This code is quite useless, but I’ve intentionally used this example so not to distract from the point.

The next example of this pattern, hopefully, shows how this can be useful. A few weeks ago, I was working with a repeater control in an asp.net site. The repeater contained a usercontrol with multiple grids and I needed easy access to manipulate every instance of this usercontrol in the repeater. So I whipped up a quick extension method for Repeater controls modeled on the IList ForEach extension:


        public static void ForEach<TControl>(this Repeater repeater, Action<TControl> action) where TControl : Control
        {
            foreach (RepeaterItem item in repeater.Items)
            {
                foreach (Control control in item.Controls)
                {
                    Repeater nestedRepeater = control as Repeater;
                    if(nestedRepeater != null)
                    {
                        nestedRepeater.ForEach(action);
                    }

                    TControl t = control as TControl;
                    if(t != null)
                    {
                        action(t);
                    }
                }
            }
        }

This method has a type parameter of TControl, which is constrained to types derived from “Control”. This is the type of the control that you are wishing to gain access to, within the repeater. It iterates the RepeaterItems of the repeater, and nested in this, it iterates the controls of each RepeaterItem. Within the nested loop, it first performs a safe cast to typeof(Repeater), and performs a recursive call if the cast is successful. Finally, a safe cast is performed to TControl, and if successful it calls the Action delegate, passing in the instance of TControl as the argument.

In a nutshell, it performs recursion to find all instances of the type TContol within the repeaters nested controls, then performs the action, specified by the consumer, on the control.

The signature of Action is a method that accepts a paramter of type TControl and returns void:


void Action(TControl control);

As the ForEach is modelled on ILists ForEach, its usage should be fairly intuitive. For example, lets say the control we are after has a public method “LoadReport”, and we’d like to call this method for each instance inside the repeater, we can call it like this:

    repeater.ForEach(control => control.LoadReport());

Piece of cake. Now as I wasn’t happy leaving this nested loop, essentially hiding the intent of code, I’ve refactored it a little, breaking down into 4 smaller methods:

        public static void ForEach<T>(this Repeater repeater, Action<T> action) where T : class
        {
            repeater.ForEachNestedRepeater(action);
            foreach (T t in repeater.GetItemsOfType<T>())
            {
                action(t);
            }
        }

        private static void ForEachNestedRepeater<T>(this Repeater repeater, Action<T> action) where T : class 
        {
            repeater.GetItemsOfType<Repeater>()
                    .ForEach(r => r.ForEach(action));
        }

        private static IEnumerable<T> GetItemsOfType<T>(this Repeater repeater)
        {
            return repeater.Items.Cast<RepeaterItem>()
                                 .Select(item => item.Controls)
                                 .OfType<T>();
        }

        private static void ForEach(this IEnumerable<Repeater> repeaters, Action<Repeater> action)
        {
            foreach (var repeater in repeaters)
            {
                action(repeater);
            }
        }

Happy coding!

Kentico vs ValidateRequest

It seems Kentico are insisting on not encoding html sent to the server from the fckEditor. Instead, Kentico advises to turn ValidateRequest off.

That’s pretty fckEd. In this post, Kentico assure us they take security seriously;

we do a lot of investigating on this issue and fix every potentially dangerous part of the system during development stage as well as later during testing stage”

It would seem to me that encoding html from the fckEditor, in order to allow ValidateRequest not to break, would be high on their list. Apparently not.

The issue I have is, not all of the website in question is the work of kentico. In fact, the bulk of the solution is custom built, while using Kentico to leverage it’s content management. Obviously then, it would be unwise to turn off ValidateRequest for the whole site.

So enough ranting about Kentico, here’s the solution. Short of hacking the entire Kentico site to htmlencode all instances of the fckEditor, we can localise turning off ValidateRequest just to Kentico pages via directory-level web.configs and keep ValidateRequest = true at the root web.config.

Add this section to Directory level web.configs for all things kentico (ie. CMSAdminControls, CMSDesk, CMSFormControls, CMSMasterPages, CMSModules, CMSPages, CMSSiteManager)

<system.web>
 <pages ValidateRequest="false" />
</system.web>

Kentico have advised they will look at fixing this issue in later releases. However, I wouldn’t hold your breath, they’ve been saying this since 2006.

So, in short, Kentico are fckEd.