Friday, February 15, 2008

SOA monitoring and discovery system

SOA is a great idea, taking chunks from EAI lessons learned, and generalising them. It occurs to me that tools for building maps of service invocations is necessary and "good". The question becomes, fairly quickly, "how?".

Asking developers and designers to be good citizens and update spreadsheets (or more sophisticated tools) doesn't work, because over a little time, the manual update system breaks down for lots of reasons (not least laziness). I suggest the only real option is to automate discovery of service invocations by watching the service interfaces themselves.

Point-to-point transports (usually TCP-based, like HTTP) make capturing traffic quite awkward, but the content can usually be extracted as it will pass through so many network elements (including the service's host machine) where a wire tap can be taken. If the communication is encrypted in-flight (as is expected for sensitive data), then an application level wiretap will be required.

BEA's AquaLogic took a very active role in this monitoring when I grappled with it; essentially as a proxy in the whole process. Maybe they made less intrusive options available in version 3... My first outing with AquaLogic didn't encourage a second try. All-in-all AquaLogic just seemed an order of magnitude too big and clunky.

I favour a distributed system of agents suitable for capturing each service invocation. Each agent could feed (largely unprocessed) data back to a cluster of analysis machines. At the end, I'd hope to see a bunch of query/display/integration tools to leverage (what would be in my experience) the only up-to-the-minute source of information about how the organisation actually hangs together.

A JMS wiretap (vendor specific, or fall-back to browsing the queues), could be implemented easily enough. For HTTP snooping, tcpdump's ok. For HTTPS, web server modules would be required (Apache, IIS?, ligHTTP?).

The analysis? Another day...

No comments: