Wednesday, 5 March 2014

Dumb clients are dumb, intelligent networks are dumber

Best film ever!
So you need process A to talk to process B, but you want asynchronicity, reliability, decoupling blah blah, and you go to the interweb, and they say "Message Queue!" a million times.

Now since SOA was just a marketing term for EAI back in the bad-old-days of the late '90s, early '00's, this was valid. Clients really were dumb, because they'd never had to talk to anyone else before, and they weren't about to change now.

Then we built new stuff, with ESB's, adding business logic on top (which was a re-imagining of transaction controllers like Tuxedo), and someone said "Message Queues!", and away we went again. But hold on there. Something has changed. We don't have to be dumb.

The term "guerrilla SOA" turned up. Which was nice, but didn't seem to make the point, that just as languages all eventually become a crap version of lisp, so nodes must eventually stop being so incredibly stupid.

Now you might want messaging. Nothing wrong with decoupling. Nothing wrong with reliable transport. Nothing wrong with broadcast semantics (topics!). Nothing wrong with rate adaptation (ok, there might be something wrong here). But these are transport level concerns, just like TCP/IP and all that stuff.

I'm not saying you can't use these for simplistic fire and forget asynchronous RPC, but it will run out of road very quickly. And that's ignoring the extra complexity. Which is not to be dismissed.

It all starts here

If you need to know if an asynchronous call has happened, and act according to the results, then you need to remember you have sent the message, and you need some kind of task that can check on the result.

If you have an extended retry interval, or simply have messages stick around for any length of time, then you need to be able to query and potentially modify the list of outstanding messages.

If you need to inspect and deal with messages that never succeed in being sent, then you need to handle failures and inspect those failures for modification and retry. So what people end up with is this:

Panzer tank architecture

Now the client is smart in that it can asynchronously (probably on an hourly schedule or similar) check the sent DB for stuck messages and act accordingly. As it's a big database very long resend intervals are okay.

The destination may also be smart which allows us to do clever things with extended multistep flows.

At this point, things are looking complicated and redundant. So the obvious simplification is this:
Chocks away!

You still have the decoupling, transport/routing and so on. It will even handle simple retry if you want.

Error handling is completely up to the client.

Since the workflow state is held in a database, this is safe.

Since the non-failure route doesn't hit a database, it is very fast (like 100 times faster).

Now, suppose you don't need decoupled transport/routing e.g. you are calling a known client. Then we can do this:
Zen architecture

This is the guerrilla web services SOA model. No need for message queues because we are handling the transport and retry in the client. And we would be doing that anyway because we need intelligence, transparency and control in the client.

Obviously I've missed out all the reliability and scaling stuff, but that is really normal webby stuff. DNS. Load balancers. I'll leave that as an exercise for the reader ;).

And so we come full circle. Message queues are for dumb clients. Add some smarts, and the need goes away. And you need smarts to retain intelligence, transparency and control in your real business logic.

Normal irreverence will resume shortly.

No comments:

Post a Comment

Whaddya think?