NServiceBus, Messaging and RESTful Endpoints

I write a lot of throw away console apps when performing system integration, especially ones that send messages on the bus to isolate sub processes for testing. This sometimes evolves into handy testing harnesses, but most of the time it just gets overwritten with new tests and later ditched.

In the past I’ve written generic http endpoints that accept commands and put them on the bus for you. I’d then use something like PostMan to post json messages to it. This is good, but you lose the help from intellisense when constructing commands, and frankly constructing these messages into JSON is a real pain.

The next step for a generic http endpoint like this is to make it more RESTful. Make it discoverable using links, after all, if it doesn’t have hyperlinks.. it’s not REST.

The idea is to have the API allow you to navigate endpoints and it’s messages as resources. In this sense the state of the endpoint might be the contracts it exposes. The API will describe the contracts and allow you to create instances of them which are put on to the bus.

To achieve this, we need to build up a list of endpoints and the contracts.. something like this

        new Endpoints()
            .ForEndpoint("Billing")
                .LookIn(typeof(Billing).Assembly)
                .For(t => IsContract(t))                       

            .ForEndpoint("Sales")
                .LookIn(typeof(Sales).Assembly)
                .For(t => IsContract(t))

            .ForEndpoint("Clients")
                .LookIn(typeof(Clients).Assembly)
                .For(t => IsContracts(t)                       
             )
        ));

This will be used by the EndpointsController to then build up a view of all the endpoints and their contracts in a RESTFul manner.

    public class EndpointController : ApiController
    {
        private readonly Endpoints endpoints;

        public EndpointController(Endpoints endpoints)
        {
            this.endpoints = endpoints;
        }

        public HttpResponseMessage Get()
        {
            return Request.CreateResponse(endpoints.AllEndpoints());
        }

        public HttpResponseMessage Get(string endpointName)
        {
            return Request.CreateResponse(endpoints.ContractsIn(endpointName));
        }

        public HttpResponseMessage Get(string endpointName, string contractName)
        {
            return Request.CreateResponse(endpoints.FindContract(endpointName, contractName));
        }

        public HttpResponseMessage Post(string endpointName, string contractName, FormDataCollection formData)
        {
            var descriptor = endpoints.FindContract(endpointName, contractName);
            var contract = Activator.CreateInstance(descriptor.Type);            

            foreach (var property in descriptor.Type.GetProperties())
            {
                var propertyValue = formData[property.Name];     

                object value = null;
                if (propertyValue.TryConvert(property.PropertyType, out value))
                {
                    property.SetValue(contract, value);
                }
                else
                {
                    descriptor.ResponseMessage = string.Format("Value for '{0}' provided was in an incorrect format.", property.Name);
                    return Request.CreateResponse(System.Net.HttpStatusCode.BadRequest, descriptor);
                }
            }

            return Request.CreateResponse(descriptor);
        }
    }

From this we can navigate the API through each endpoint, easily identify which contracts belong to which endpoint and build a form to post them to the API.

Kudo’s must be given here to Howard Dierking for RestBugs. This is a great example of a Restful API, using a RazorMediaTypeFormatter to respresnt resources as html.

Using a html representation will allows us to interact with the API in a browser, just like you might with a typical website. Ironically, your endpoints begin to look a lot like asmx services when navigating to them in the browser. As a note, building REST APIs like this, i.e. that are browsable, is a very powerful tool for making sure clients can easily navigate the API by following links. (For a great series on REST have a look at Ploeh’s thoughts on the topic.)

Let’s have a look, the root of the API lists out all the endpoints:

API Root displays all endoints

Selecting an endpoint will list out all the contracts for that endpoint..

Selecting endpoint displays its contracts

Clicking on a contract will present a form, allowing you to post a command to the endpoint.

API returns form for posting a contract.

Simples.

The source for this is up on Git here. At the moment it’s really just a proof of concept and doesn’t actually post messages on any bus, my thoughts on this are to try to keep it agnostic to any kind of messaging infrastructure. I use NServiceBus in my day job, but it really could be used for anything. Most likely I’ll allow the “Endpoints” class to capture actions for bus.Send / bus.Publish depending on whether a message is a command or an event.

Feel free to fork what’s up there now and get it working with your messaging tooling!

ASP.NET WebApi – Breaking change to MediaTypeFormatters

In my preview post, I described an implementation that would allow multipart/form-data image uploads to be binded to an ImageMedia model by implementing the MediaTypeFormatter. That post was based on the beta api and has since been changed a few times.

In the previous post, the method signature for ReadFromStreamAsync was:

        protected async override Task OnReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            FormatterContext formatterContext)

This allowed us to get at the request via the FormatterContext like this:

            HttpRequestMessage request = formatterContext.Request;

A change was then made that meant we could no longer access the HttpRequestMessage, and therefore no longer access the HttpContent property of the request.

Essentially, this change removed the FormatterContext parameter, and in its place passed an IFormatterLogger:

        public async override Task ReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            IFormatterLogger formatterLogger)

On the 15 August 2012, the ASP.NET WebApi was Released , and with it came a welcome change.

The team decided to replace the HttpContentHeaders parameter with the HttpContent allowing access to all the extension methods that come with it. The implementation I provided, depended on ReadAsMultipartAsync.

This method itself has also changed, as it no longer returns a collection of HttpContent, and now returns MultipartMemoryStreamProvider . The provider then allows access to the HttpContent collection via the Contents parameter.

Phew! So after all that, we can now have our ImageMediaTypeFormatter extract our image stream once again.

TL;DR
Enough of the pleasantries, here’s the updated ReadFromStreamAsync implementation.


        public async override Task<object> ReadFromStreamAsync(
                Type type, Stream stream, HttpContent content,
                IFormatterLogger formatterLogger)
        {
            if (!content.IsMimeMultipartContent())
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var provider = await content.ReadAsMultipartAsync();

            var mediacontent = provider.Contents.First(x =>
                SupportedMediaTypes.Contains(x.Headers.ContentType));

            string fileName = mediacontent.Headers.ContentDisposition.FileName;
            string mediaType = mediacontent.Headers.ContentType.MediaType;

            using (var imgstream = await mediacontent.ReadAsStreamAsync())
            {
                byte[] imagebuffer = imgstream.ReadFully();
                return new ImageMedia(fileName, mediaType, imagebuffer);
            }
        }

ASP.NET WebApi upload image with custom MediaTypeFormatter

UPDATE: changes to WebApi mean this code no longer works, see here for update.

I’ve been reading a few tutorials on www.asp.net and come across a good article demonstrating the basics of MediaTypeFormatters.

I’m currently working on a personal project utilising WebApi to post an image for OCR. I don’t need to save this file, I just need a byte array of it to perform some character recognition using Tesseract. One of the tutorials on http://www.asp.net, Forms and Multi-part MIME, describes how to upload an image by reading the multipart data directly inside the controller, and saving it to appdata folder.

This doesn’t seem ideal for my scenario, firstly, I don’t need to store the images and, more importantly, it makes it difficult to test.

Ideally, I needed my actions signature to be like this:

[HttpPost]
public Task<string> Upload(ImageMedia media)

Where ImageMedia looks like this:

    public class ImageMedia
    {
        public string FileName { get; set; }
        public string MediaType { get; set; }
        public byte[] Buffer { get; set; }

        public ImageMedia(string fileName, string mediaType, byte[] imagebuffer)
        {
            FileName = fileName;
            MediaType = mediaType;
            Buffer = imagebuffer;
        }
    }

Now, implementing a new MediaTypeFormatter I can extract the image and populate my ImageMedia model.

Firstly, we need to to tell the framework what media types the MediaFormatter applies to, we can do this by adding MediaTypeHeaderValues to the list of SupportedMediaTypes inside the constructor, something like this does the job:

        public ImageMediaFormatter()
        {
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpeg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/png"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("multipart/form-data"));
        }

We also need to specifiy what model types the formatter can write/read, because I’m only reading from the stream to my ImageMedia, I’ll need to override CanReadType comparing the Type passed in to typeof(ImageMedia):

        protected override bool CanReadType(Type type)
        {
            return type == typeof(ImageMedia);
        }

And I’ll simply return false from CanWriteType:

        protected override bool CanWriteType(Type type)
        {
            return false;
        }

Now for the actual work:

1. First check that the request is IsMimeMultipartContent
2. Then read the content of the request as Multipart.
3. We the find the part that that matches one of our SupportedMediaTypes, this will be the image we’re after.
4. We extract te fileName and mediaType from the headers of the content we retrieved.
5. then on content call ReadAsStreamAsync()
6. as ImageMedia wants a byte[] I call a handy helper “ReadFully” which is an extension method on Stream, (written by Jon Skeet)

       protected async override Task OnReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            FormatterContext formatterContext)
        {
            HttpRequestMessage request = formatterContext.Request;
            if (!request.Content.IsMimeMultipartContent())
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var parts = await request.Content.ReadAsMultipartAsync();
            var content = parts.First(x =>
                SupportedMediaTypes.Contains(x.Headers.ContentType));

            string fileName = content.Headers.ContentDisposition.FileName;
            string mediaType = content.Headers.ContentType.MediaType;

            using (var imgstream = await content.ReadAsStreamAsync())
            {
                byte[] imagebuffer = imgstream.ReadFully();
                return new ImageMedia(fileName, mediaType, imagebuffer);
            }
        }

And Jon Skeets read fully:

        public static byte[] ReadFully(this Stream input)
        {
            byte[] buffer = new byte[16 * 1024];
            using (MemoryStream ms = new MemoryStream())
            {
                int read;
                while ((read = input.Read(buffer, 0, buffer.Length)) &gt; 0)
                {
                    ms.Write(buffer, 0, read);
                }
                return ms.ToArray();
            }
        }

And thats it!!! So now the controller is no longer responsible for extracting the content of the request, and it can be left with simply coordinating it. Our ImageMediaFormatter will kick in any time an action on our WebApi requires an ImageMedia passed in.

For clarity, here’s the ImageMediaFormatter in full:


 public class ImageMediaFormatter : MediaTypeFormatter
    {
        public ImageMediaFormatter()
        {
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpeg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/png"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("multipart/form-data"));
        }

        protected override bool CanReadType(Type type)
        {
            return type == typeof(ImageMedia);
        }

        protected override bool CanWriteType(Type type)
        {
            return false;
        }

        protected async override Task OnReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            FormatterContext formatterContext)
        {
            HttpRequestMessage request = formatterContext.Request;
            if (!request.Content.IsMimeMultipartContent())
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var parts = await request.Content.ReadAsMultipartAsync();
            var content = parts.First(x =>
                SupportedMediaTypes.Contains(x.Headers.ContentType));

            string fileName = content.Headers.ContentDisposition.FileName;
            string mediaType = content.Headers.ContentType.MediaType;

            using (var imgstream = await content.ReadAsStreamAsync())
            {
                byte[] imagebuffer = imgstream.ReadFully();
                return new ImageMedia(fileName, mediaType, imagebuffer);
            }
        }
    }