Processing the messages is generally going to be an in-memory operation - unless your machine is very short on memory the limiting factor is going to be the CPU. Since Chameleon is extremely fast, in practice this is not the limiting factor in an interface. The place that needs to be optimized is your input/output via the network and disc.

A good queuing system should allow you to request the retrieval of several messages at a time since there might be a bottleneck involving reading from a disk drive.

When you are using a queue, the processing of the messages does not have to be in the same application. It can be a completely independent process, just like an assembly line worker.

Multi-threading at this point might be useful if you have this situation:

• There are several destinations for messages;
• Some messages just need to be passed through as is; and
• Some transformations are much slower than others.

In this type of situation it would make sense to have one thread that takes the chunks of data from the queue. It then assembles each message by using the LLP protocol to work out the start and end of each message.

Each message should then be categorized as to where it should be sent, and if and how it should be transformed. For details on how to deal with this situation, see Dealing with Multiple Vendors.

The messages can then be dispatched to in-memory queues (large string buffers for speed), which can then be handled by one thread for each type. This will take care of transforming the message if required, then putting the messages into separate queues for each part. Note that it will be good if your queuing mechanism has some kind of callback function that can notify your application when a message has been successfully committed to the queue. This can then be used to purge messages from the original input queue.