NServiceBus, Messaging and RESTful Endpoints

July 20, 2014 Leave a comment

I write a lot of throw away console apps when performing system integration, especially ones that send messages on the bus to isolate sub processes for testing. This sometimes evolves into handy testing harnesses, but most of the time it just gets overwritten with new tests and later ditched.

In the past I’ve written generic http endpoints that accept commands and put them on the bus for you. I’d then use something like PostMan to post json messages to it. This is good, but you lose the help from intellisense when constructing commands, and frankly constructing these messages into JSON is a real pain.

The next step for a generic http endpoint like this is to make it more RESTful. Make it discoverable using links, after all, if it doesn’t have hyperlinks.. it’s not REST.

The idea is to have the API allow you to navigate endpoints and it’s messages as resources. In this sense the state of the endpoint might be the contracts it exposes. The API will describe the contracts and allow you to create instances of them which are put on to the bus.

To achieve this, we need to build up a list of endpoints and the contracts.. something like this

        new Endpoints()
            .ForEndpoint("Billing")
                .LookIn(typeof(Billing).Assembly)
                .For(t => IsContract(t))                       

            .ForEndpoint("Sales")
                .LookIn(typeof(Sales).Assembly)
                .For(t => IsContract(t))

            .ForEndpoint("Clients")
                .LookIn(typeof(Clients).Assembly)
                .For(t => IsContracts(t)                       
             )
        ));

This will be used by the EndpointsController to then build up a view of all the endpoints and their contracts in a RESTFul manner.

    public class EndpointController : ApiController
    {
        private readonly Endpoints endpoints;

        public EndpointController(Endpoints endpoints)
        {
            this.endpoints = endpoints;
        }

        public HttpResponseMessage Get()
        {
            return Request.CreateResponse(endpoints.AllEndpoints());
        }

        public HttpResponseMessage Get(string endpointName)
        {
            return Request.CreateResponse(endpoints.ContractsIn(endpointName));
        }

        public HttpResponseMessage Get(string endpointName, string contractName)
        {
            return Request.CreateResponse(endpoints.FindContract(endpointName, contractName));
        }

        public HttpResponseMessage Post(string endpointName, string contractName, FormDataCollection formData)
        {
            var descriptor = endpoints.FindContract(endpointName, contractName);
            var contract = Activator.CreateInstance(descriptor.Type);            

            foreach (var property in descriptor.Type.GetProperties())
            {
                var propertyValue = formData[property.Name];     

                object value = null;
                if (propertyValue.TryConvert(property.PropertyType, out value))
                {
                    property.SetValue(contract, value);
                }
                else
                {
                    descriptor.ResponseMessage = string.Format("Value for '{0}' provided was in an incorrect format.", property.Name);
                    return Request.CreateResponse(System.Net.HttpStatusCode.BadRequest, descriptor);
                }
            }

            return Request.CreateResponse(descriptor);
        }
    }

From this we can navigate the API through each endpoint, easily identify which contracts belong to which endpoint and build a form to post them to the API.

Kudo’s must be given here to Howard Dierking for RestBugs. This is a great example of a Restful API, using a RazorMediaTypeFormatter to respresnt resources as html.

Using a html representation will allows us to interact with the API in a browser, just like you might with a typical website. Ironically, your endpoints begin to look a lot like asmx services when navigating to them in the browser. As a note, building REST APIs like this, i.e. that are browsable, is a very powerful tool for making sure clients can easily navigate the API by following links. (For a great series on REST have a look at Ploeh’s thoughts on the topic.)

Let’s have a look, the root of the API lists out all the endpoints:

API Root displays all endoints

Selecting an endpoint will list out all the contracts for that endpoint..

Selecting endpoint displays its contracts

Clicking on a contract will present a form, allowing you to post a command to the endpoint.

API returns form for posting a contract.

Simples.

The source for this is up on Git here. At the moment it’s really just a proof of concept and doesn’t actually post messages on any bus, my thoughts on this are to try to keep it agnostic to any kind of messaging infrastructure. I use NServiceBus in my day job, but it really could be used for anything. Most likely I’ll allow the “Endpoints” class to capture actions for bus.Send / bus.Publish depending on whether a message is a command or an event.

Feel free to fork what’s up there now and get it working with your messaging tooling!

Emotiv EPOC and the Event Store – Streams of Consciousness

February 27, 2013 Leave a comment

I’ve found a way to combine two of my interests; the Emotiv EPOC headset and the Event Store. The EPOC is “a revolutionary personal interface for human computer interacton”. Couple this with “The awesome, rock-solid, super-fast persistence engine for event-sourced applications” that is the Event Store and you have:

Streams of Consciousness

The project is just a way to learn both of these technologies whilst creating something that, I think, is pretty cool.

The idea? To send the stream of events from the EPOC headset into the Event Store! I then re-use (steal) the event stores health chart graphs to display real-time data of the events measuring my current level of engagement, excitement/boredom, frustration and meditation as they are being read by the EPOC headset – all in JavaScript!!!

ouroepoc

I’ve essentially just reverse engineered the example chat room and charts to build the charts. It’s quite simple to consume the event store using the projections.js javascript that powers the chat room.

After digging into the way the chat works, I mashed up this javascript that will build the stats for the graphs and publish that to the onStateUpdated callback.

function emoProjection(options) {
    var streamName = options.streamName || "";
    var onStateUpdated = options.onStateUpdated || function () { };

    return es.projection({
        body: function () {
            fromStream(streamName).when({
                'AffectivEmoStateUpdated': function (state, event) {
                    var body = event.body;
                    return {
                        "StateEvent" : "AffectiveEmoStateUpdated",
                        "Excitement Short Term Score": body.ExcitementShortTermScore,
                        "Excitement Long Term Score": body.ExcitementLongTermScore,
                        "Frustration Score": body.FrustrationScore,
                        "Engagement Boredom Score": body.EngagementBoredomScore,
                        "Meditation Score": body.MeditationScore

                    };
                }
            });
        },
        onStateUpdate: function (state) {
            onStateUpdated(state);
        },
        showError: function (err) {
            $('.error').text(err);
        },
        hideError: function () {
            $('.error').hide().text("");
        }
    });
}

To consume this projection, new it up and assign the function that updates the charts to the onStateUpdated callback.

  var projection = new emoProjection({
            streamName: streamName,
            onStateUpdated: function (state) { updateCharts(state);}
        });

        projection.start();

Bob’s your uncle!

Below are screen prints of the charts capturing my current state while I’m wearing the EPOC headset and writing this post!

Event Charts

Excitement Short Term Score

Next steps: I’ll get these charts up on a live server soon and then stream my consciousness into the interwebs for all to see! Hopefully I’ll get the chance to blog about it too.

Categories: TheEventStore Tags: , ,

ASP.NET WebApi – Breaking change to MediaTypeFormatters

September 23, 2012 3 comments

In my preview post, I described an implementation that would allow multipart/form-data image uploads to be binded to an ImageMedia model by implementing the MediaTypeFormatter. That post was based on the beta api and has since been changed a few times.

In the previous post, the method signature for ReadFromStreamAsync was:

        protected async override Task OnReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            FormatterContext formatterContext)

This allowed us to get at the request via the FormatterContext like this:

            HttpRequestMessage request = formatterContext.Request;

A change was then made that meant we could no longer access the HttpRequestMessage, and therefore no longer access the HttpContent property of the request.

Essentially, this change removed the FormatterContext parameter, and in its place passed an IFormatterLogger:

        public async override Task ReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            IFormatterLogger formatterLogger)

On the 15 August 2012, the ASP.NET WebApi was Released , and with it came a welcome change.

The team decided to replace the HttpContentHeaders parameter with the HttpContent allowing access to all the extension methods that come with it. The implementation I provided, depended on ReadAsMultipartAsync.

This method itself has also changed, as it no longer returns a collection of HttpContent, and now returns MultipartMemoryStreamProvider . The provider then allows access to the HttpContent collection via the Contents parameter.

Phew! So after all that, we can now have our ImageMediaTypeFormatter extract our image stream once again.

TL;DR
Enough of the pleasantries, here’s the updated ReadFromStreamAsync implementation.


        public async override Task<object> ReadFromStreamAsync(
                Type type, Stream stream, HttpContent content,
                IFormatterLogger formatterLogger)
        {
            if (!content.IsMimeMultipartContent())
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var provider = await content.ReadAsMultipartAsync();

            var mediacontent = provider.Contents.First(x =>
                SupportedMediaTypes.Contains(x.Headers.ContentType));

            string fileName = mediacontent.Headers.ContentDisposition.FileName;
            string mediaType = mediacontent.Headers.ContentType.MediaType;

            using (var imgstream = await mediacontent.ReadAsStreamAsync())
            {
                byte[] imagebuffer = imgstream.ReadFully();
                return new ImageMedia(fileName, mediaType, imagebuffer);
            }
        }
Categories: ASP.NET, C#, WepApi

ASP.NET WebApi upload image with custom MediaTypeFormatter

June 3, 2012 8 comments

UPDATE: changes to WebApi mean this code no longer works, see here for update.

I’ve been reading a few tutorials on www.asp.net and come across a good article demonstrating the basics of MediaTypeFormatters.

I’m currently working on a personal project utilising WebApi to post an image for OCR. I don’t need to save this file, I just need a byte array of it to perform some character recognition using Tesseract. One of the tutorials on http://www.asp.net, Forms and Multi-part MIME, describes how to upload an image by reading the multipart data directly inside the controller, and saving it to appdata folder.

This doesn’t seem ideal for my scenario, firstly, I don’t need to store the images and, more importantly, it makes it difficult to test.

Ideally, I needed my actions signature to be like this:

[HttpPost]
public Task<string> Upload(ImageMedia media)

Where ImageMedia looks like this:

    public class ImageMedia
    {
        public string FileName { get; set; }
        public string MediaType { get; set; }
        public byte[] Buffer { get; set; }

        public ImageMedia(string fileName, string mediaType, byte[] imagebuffer)
        {
            FileName = fileName;
            MediaType = mediaType;
            Buffer = imagebuffer;
        }
    }

Now, implementing a new MediaTypeFormatter I can extract the image and populate my ImageMedia model.

Firstly, we need to to tell the framework what media types the MediaFormatter applies to, we can do this by adding MediaTypeHeaderValues to the list of SupportedMediaTypes inside the constructor, something like this does the job:

        public ImageMediaFormatter()
        {
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpeg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/png"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("multipart/form-data"));
        }

We also need to specifiy what model types the formatter can write/read, because I’m only reading from the stream to my ImageMedia, I’ll need to override CanReadType comparing the Type passed in to typeof(ImageMedia):

        protected override bool CanReadType(Type type)
        {
            return type == typeof(ImageMedia);
        }

And I’ll simply return false from CanWriteType:

        protected override bool CanWriteType(Type type)
        {
            return false;
        }

Now for the actual work:

1. First check that the request is IsMimeMultipartContent
2. Then read the content of the request as Multipart.
3. We the find the part that that matches one of our SupportedMediaTypes, this will be the image we’re after.
4. We extract te fileName and mediaType from the headers of the content we retrieved.
5. then on content call ReadAsStreamAsync()
6. as ImageMedia wants a byte[] I call a handy helper “ReadFully” which is an extension method on Stream, (written by Jon Skeet)

       protected async override Task OnReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            FormatterContext formatterContext)
        {
            HttpRequestMessage request = formatterContext.Request;
            if (!request.Content.IsMimeMultipartContent())
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var parts = await request.Content.ReadAsMultipartAsync();
            var content = parts.First(x =>
                SupportedMediaTypes.Contains(x.Headers.ContentType));

            string fileName = content.Headers.ContentDisposition.FileName;
            string mediaType = content.Headers.ContentType.MediaType;

            using (var imgstream = await content.ReadAsStreamAsync())
            {
                byte[] imagebuffer = imgstream.ReadFully();
                return new ImageMedia(fileName, mediaType, imagebuffer);
            }
        }

And Jon Skeets read fully:

        public static byte[] ReadFully(this Stream input)
        {
            byte[] buffer = new byte[16 * 1024];
            using (MemoryStream ms = new MemoryStream())
            {
                int read;
                while ((read = input.Read(buffer, 0, buffer.Length)) &gt; 0)
                {
                    ms.Write(buffer, 0, read);
                }
                return ms.ToArray();
            }
        }

And thats it!!! So now the controller is no longer responsible for extracting the content of the request, and it can be left with simply coordinating it. Our ImageMediaFormatter will kick in any time an action on our WebApi requires an ImageMedia passed in.

For clarity, here’s the ImageMediaFormatter in full:


 public class ImageMediaFormatter : MediaTypeFormatter
    {
        public ImageMediaFormatter()
        {
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpeg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/jpg"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("image/png"));
            SupportedMediaTypes.Add(new MediaTypeHeaderValue("multipart/form-data"));
        }

        protected override bool CanReadType(Type type)
        {
            return type == typeof(ImageMedia);
        }

        protected override bool CanWriteType(Type type)
        {
            return false;
        }

        protected async override Task OnReadFromStreamAsync(
            Type type, Stream stream, HttpContentHeaders contentHeaders,
            FormatterContext formatterContext)
        {
            HttpRequestMessage request = formatterContext.Request;
            if (!request.Content.IsMimeMultipartContent())
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var parts = await request.Content.ReadAsMultipartAsync();
            var content = parts.First(x =>
                SupportedMediaTypes.Contains(x.Headers.ContentType));

            string fileName = content.Headers.ContentDisposition.FileName;
            string mediaType = content.Headers.ContentType.MediaType;

            using (var imgstream = await content.ReadAsStreamAsync())
            {
                byte[] imagebuffer = imgstream.ReadFully();
                return new ImageMedia(fileName, mediaType, imagebuffer);
            }
        }
    }
Categories: ASP.NET, C#, WepApi Tags: ,

POCO’s are allowed to have behaviour…

May 28, 2012 1 comment

…in fact they probably should. I was recently told that logic shouldn’t be put into the domain because the domain objects are simple POCO’s. This seems to be a misunderstanding of the term.

POCO is derived from the term POJO, first defined by Martin Fowler et al. A POJO (Plain Old Java Object) is simply an object that has no dependency on an external framework, a backlash to EJB’s and heavy fraweworks that enforced inheritance.

POCO’s, like POJOs never precluded behaviour. In fact, POCO’s without behaviour are commonly referred to as “Anemic”, an anti pattern described by Fowler here.

In general, the more behavior you find in the services, the more likely you are to be robbing yourself of the benefits of a domain model. If all your logic is in services, you’ve robbed yourself blind.

This doesn’t mean Anemic Models are all bad, as Jimmy Bogard writes here: “Context is King” and sometimes a “Persistent Model” is sufficient to meet the systems needs.

So when thinking about modeling your domain, don’t simply disregard modeling it’s behaviour because they are only “simple POCO’s”, you might be robbing yourself blind.

Categories: Uncategorized

Integration Tests – Tests Fail When Network Unavailable

May 11, 2012 Leave a comment

I code a lot whilst travelling to work on the train. One thing that has bugged me is integration tests that rely on remote services will fail unless I’ve tethered my phone. Having those few red tests littering my test results kind of irks me and leads me to run the tests less often or not at all.

What I wanted was an ignore attribute that could detect if the network was unavailable and ignore the test, but I couldn’t find anythign to suit. As there isn’t great support for extending test attributes, I decided to build support in to my IntergrationBaseTest class.

This could have been solved easily by placing these two lines at the start of my test:

            
           if(!NetworkInterface.GetIsNetworkAvailable())
                Assert.Inconclusive();

The above is simple but I wanted to explore the idea of custom test attributes to see what I could come up with.

What first sprang to mind was ASP.NET MVC’s ActionFilterAttributes. MSDN describes ActionFilterAttributes as the following:

ActionFilterAttribute is an abstract class that has four virtual methods that you can override: OnActionExecuting, OnActionExecuted, OnResultExecuting, and OnResultExecuted.

The ASP.NET MVC framework will call the OnActionExecuting method of your action filter before it calls any action method that is marked with your action filter attribute. Similarly, the framework will call the OnActionExecuted method after the action method has finished.

Taking this approach and applying it to MSTest I created the below abstract attribute:

  
    public abstract class CustomTestAttribute : Attribute
    {
        public virtual void OnTestExecuting(TestContext context) {}
        public virtual void OnTestExecuted(TestContext context) {}
    }

I marked the methods as virtual so as not to force derived classes to implement both. As you might have guessed the OnTestExecuting maps to methods marked with the TestInitializeAttribute and the OnTestExecuted maps to the TestCleanupAttribute in MSTest. The benefit we gain here is the ability to descriminate down to the method level within a class.

To support the new attribute the base class is going to have to know about it and execute the respective methods in the corresponding initialize and cleanup methods.

First, to deal with multiple attributes as a single unit, I created a CompositeTestAttribute, deriving form the CustomTestAttribute:

  
    public sealed class CompositeTestAttribute : CustomTestAttribute
    {
        private readonly IEnumerable<CustomTestAttribute> attributes;

        public CompositeTestAttribute(IEnumerable<CustomTestAttribute> attributes)
        {
            this.attributes = attributes;
        }

        public override void OnTestExecuting(TestContext context)
        {
            foreach (var customTestAttribute in attributes)
            {
                customTestAttribute.OnTestExecuting(context);
            }
        }

        public override void OnTestExecuted(TestContext context)
        {
            foreach (var customTestAttribute in attributes)
            {
                customTestAttribute.OnTestExecuting(context);
            }
        }
    }

To determine which attributes the executing test is decorated with we can use the TestContext which gets injected by mstest at runtime, all you need to do is dangle a property on the base class:

  
	public TestContext TestContext { get; set; }

Using the TestContext we can access the “FullyQualifiedTestClassName” and “TestName” properties. This gives us enough information to generate a type and access the attributes:

  
        private IEnumerable<CustomTestAttribute> GetCustomTestAttributes()
        {
            IEnumerable<CustomTestAttribute> attributes = Enumerable.Empty<CustomTestAttribute>();
            string className = TestContext.FullyQualifiedTestClassName;
            string methodName = TestContext.TestName;

            Type testClassType = Type.GetType(className);
            if (testClassType != null)
            {
                MemberInfo mi = testClassType.GetMethod(methodName);
                attributes = mi.GetCustomAttributes<CustomTestAttribute>();
            }

            return attributes;
        }

All that’s needed now is to invoke OnTestExecuting and OnTestExecuted in the TestInitialize and TestCleanup methods respectively.

  
        [TestInitialize()]
        public void MyTestInitialize()
        {
            new CompositeTestAttribute(GetCustomTestAttributes())
                .OnTestExecuting(this.TestContext);
        }

        [TestCleanup()]
        public void MyTestCleanup()
        {
            new CompositeTestAttribute(GetCustomTestAttributes())
                .OnTestExecuted(this.TestContext);
        }

Now, that the infrastucture is there, we can start implementing CustomTestAttributes. Which brings me back to the reason for all this.
To determine if the Network is available, there is a handy helper method located in “System.Net.NetworkInformation.NetworkInterface” called

“GetIsNetworkAvailable()”

  
    public class IgnoreWhenNetworkUnavailableAttribute : CustomTestAttribute
    {
        public override void OnTestExecuting(TestContext context)
        {
            if(!NetworkInterface.GetIsNetworkAvailable())
                Assert.Inconclusive();
        }
    }

Now to make use of this, all we have todo is derive from the IntegrationBaseTest and decorate our network dependent test methods. :)

For clarity, below is the full code. Although this could have been achieved by far simpler means, it was an interesting exercise, that allowed me to delve into some code and have a little fun. I hope you find this helpful and put CustomTestAttributes to use in scenarios I haven’t thought of.

  
 /// <summary>
    /// Summary description for IntegrationTestBase
    /// </summary>
    [TestClass]
    public class IntegrationTestBase
    {
	public TestContext TestContext { get; set; }

        [TestInitialize()]
        public void MyTestInitialize()
        {
            new CompositeTestAttribute(GetCustomTestAttributes())
                .OnTestExecuting(this.TestContext);
        }

        [TestCleanup()]
        public void MyTestCleanup()
        {
            new CompositeTestAttribute(GetCustomTestAttributes())
                .OnTestExecuted(this.TestContext);
        }

        private IEnumerable<CustomTestAttribute> GetCustomTestAttributes()
        {
            IEnumerable<CustomTestAttribute> attributes = Enumerable.Empty<CustomTestAttribute>();
            string className = TestContext.FullyQualifiedTestClassName;
            string methodName = TestContext.TestName;

            Type testClassType = Type.GetType(className);
            if (testClassType != null)
            {
                MemberInfo mi = testClassType.GetMethod(methodName);
                attributes = mi.GetCustomAttributes<CustomTestAttribute>();
            }

            return attributes;
        }
    }

    public class IgnoreWhenNetworkUnavailableAttribute : CustomTestAttribute
    {
        public override void OnTestExecuting(TestContext context)
        {
            if(!NetworkInterface.GetIsNetworkAvailable())
                Assert.Inconclusive();
        }
    }

    public abstract class CustomTestAttribute : Attribute
    {
        public virtual void OnTestExecuting(TestContext context) {}
        public virtual void OnTestExecuted(TestContext context) {}
    }

    public sealed class CompositeTestAttribute : CustomTestAttribute
    {
        private readonly IEnumerable<CustomTestAttribute> attributes;

        public CompositeTestAttribute(IEnumerable<CustomTestAttribute> attributes)
        {
            this.attributes = attributes;
        }

        public override void OnTestExecuting(TestContext context)
        {
            foreach (var customTestAttribute in attributes)
            {
                customTestAttribute.OnTestExecuting(context);
            }
        }

        public override void OnTestExecuted(TestContext context)
        {
            foreach (var customTestAttribute in attributes)
            {
                customTestAttribute.OnTestExecuting(context);
            }
        }
    }

MVC 4 Beta Project Templates – Visual Studio Freezes on Adding Script tags to view

February 29, 2012 3 comments

I’ve been playing around with the new WebApi that was released with MVC 4 Beta and have come across what seems like a bug.

So far I’ve been able to reproduce this consistently on two separate machines.

The issue occurs when adding script tags to a view, it seems that visual studio hangs for at least 30 seconds.

Has anyone else been playing with the WebApi and noticing this issue?

UPDATE: this seems to happen in any MVC 4 project template and, in particular, when hitting the carriage return.

Categories: Uncategorized Tags:
Follow

Get every new post delivered to your Inbox.