Multiple messages and transport messages in NServiceBus

Andreas Öhlund posted recently on the concept of the “transport message” in NServiceBus. One of the mistakes I often see (and made myself) was misunderstanding the boundary of the unit of work NServiceBus applies to messages, especially around sending multiple messages.

In many of our systems, we consume flat files from third party integration partners. We take these flat files and deserialize each line of the file into a distinct message, so we first tried to do something like this:

ProcessLineInFileMessage[] messages = ConvertFileToMessages(file);

Bus.Send(messages); // Sends all logical messages in 1 transport message

The problem we hit was that the unit of work boundary in NServiceBus is around the transport message, not the logical message. In a file of a million lines, that’s a million logical messages bundled together into one single transport message, and one transaction boundary! We had assumed that the overload for sending multiple messages was simply a helper method that encapsulated a “foreach”. Well, no, it doesn’t. All the messages are wrapped in an envelope known as a “transport message”, and it’s the transport message that defines the unit of work boundary (since that’s the physical message sitting in the queue).

Needless to say, we saw database connection timeouts pretty much immediately. Instead, we modified our use of the bus to instead send one logical message per physical transport message, with our friend the “foreach”:

ProcessLineInFileMessage[] messages = ConvertFileToMessages(file);

foreach (var message in messages)
    Bus.Send(message); // Send one logical message at a time

So when would you use the overloads for sending multiple messages? I’m not sure, but I’ll update if I find out!

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Jimmy Bogard

I'm a technical architect with Headspring in Austin, TX. I focus on DDD, distributed systems, and any other acronym-centric design/architecture/methodology. I created AutoMapper and am a co-author of the ASP.NET MVC in Action books.
This entry was posted in NServiceBus. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Andreas Öhlund

    A common usage is to bundle commands from a smart client (probably a sign of bad design but whatever). bus.Send(new ChangeAddressDetailsCommand(), new MoveCustomerCommand()). This guarantees that both are processed as a unit

    • Anonymous

      If you have a per-client customized UI, like say a wizard that has varying steps this would be a good solution.

  • http://twitter.com/thefringeninja João P. Bragança

    Now you are opening and closing a million database connections. I would use batches of multiple logical messages instead.

  • Pingback: Distributed Weekly 140 — Scott Banwart's Blog

  • http://www.acensco.com Bruno Bertechini

    How do we test a Handler that Bus.Send inside a for loop ?

    Looks like .ExpectSend is called only once in this case.

    Bruno

    • jbogard

      ExpectSend? What’s that?

      • http://www.acensco.com Bruno Bertechini

        NServiceBus.Testing api (Unit Testing your handler)

        Test.Handler()
        .ExpectSendLocal(cmd =>
        {
        // Assert cmd values
        }

        This will happen if the Handler has some sort of loop to send commands, for example:

        [Handler]
        for(int i = 0; i { cmd.Id = id++; });
        }

        I have an issue open:

        https://github.com/Particular/NServiceBus/issues/2588