These notes on enterprise-level messaging applications offered by IBM, Microsoft, and other vendors. This is related to my pages on enterprise development and TIBCO. Let me know what you find helpful or missing.
Like email, also called "interpersonal messaging", enterprise message queuing applications use asynchronous processing to send and receive messages. An application can send messages when the recipient is not up and running simultaneously. This allows a component to send information to another and to continue to operate without receiving an immediate response.
Email provides a "best effort" delivery method that does NOT automatically confirm receipt.
Transactional MessagingBut transactional messaging systems also assumes the responsibility to ensure reliable delivery of messages to the target queue, or, if it cannot do so, take appropriate action.
Message queuing applications are generally designed so that messages flow in a request/reply fashion. But unlike emails, a transactional messaging client does not have to request messages in order to receive them. Inter-program communication messages are delievered to a client as they arrive.
After the messaging system accepts a message from the application into a queue as an intermediate storage point. The application is then free to continue other work. This allows communicating programs to run at different times.
Message queuing provides the connectivity layer "backbone" for what IBM calls an enterprise service bus (ESB) within a service-orientated architecture (SOA).
Messaging components read and operate on the messages independently. The sending and receiving applications are uncoupled or loosely coupled. rather than tightly coupled, such as with Remote Procedure Call (RPC). The component sending messages can send messages without depending on information about other components' interfaces. Programs are insulated from network complexities (fixed message formats, communications protocols, etc.). Communication protocols are hidden from the applications. All this allows components to be easily replaced.
Most importantly, messaging applications are designed to be reliable — ensure that a message is delivered once and only once. Lower levels of reliability are available for applications that can afford to miss messages or to receive duplicate messages.
Passing SOAP messages through message queues provide visibility and auditability to interactions.
Sun developed the Java System Message Queue Service (JMS) application-level messaging standard API SDK to enable J2EE peer-to-peer apps (using Java 2 SDK, Enterprise Edition, version 1.3) to create, send, receive, and read loosely coupled, reliable, and asynchronous messages.
The sample programs for the API 1.1 doc simplifies and improves programmer productivity by defining a common set of concepts and programming strategies that will be supported by all JMS technology-compliant messaging systems.
Sun's free "Platform Edition" has a Windows variant (now) but does not have the scalability, reliability and advanced security features of the for-fee enterprise edition.
JMS operates in two modes when calling a servlet at a JMS Destination:
JMS messages can participate in Java Transaction API (JTA) transactions.
IBM's Message Queue product (developed in IBM's Hurley Labs in the UK) has the largest market share of any messaging middleware solution. Its full name was "IBM WebSphere MQSeries" because it runs on WebSphere servers. IBM dropped the "Series" at v6.0. v7 debut Oct 2009.
The MQ product consist of three major components operating independently of each other:
Servers transmit messages communicate via channels — each a unidirectional link between a MCA (Message Channel Agent) on each queue manager at each end of the communicating channel.
MQSeries uses the Message Channel Protocol (MCP) to ensure delivery.
MQ messages can be as large as 100MB with v6.0, its latest release.
Several Queue Managers can be clustered so that each queue manager can access the queues of all other queue managers in a queue-sharing group through Cluster-sender and Cluster-receiver channels rather than remote queue definitions. Information about clusters are held in a SYSTEM CLUSTER REPOSITORY QUEUE and exchanged through a SYSTEM CLUSTER COMMAND QUEUE.
To ensure high availability and pull workload balancing, applications request web services by connecting to (targeting) a shared queue.
Instead of using a private channel definition to target a specific queue manager, a global definition in a shared repository (DB2 database available to all queue managers) specifies that all servers in the queue-sharing group listen on a generic port (e.g., 4005) for inbound network attach requests. On z/OS, this is done by a listener task started by the channel initiator on each server handling communication between WebSphere MQ on z/OS VTAM and other systems (AIX, Windows, etc.).
If outbound messages are always smaller than 63KB (including headers), a shared channel can provide high availability to communication via synchronization information stored in the SYSTEM.QSG.CHANNEL.SYNCQ queue. Private channels store synchronization data in the SYSTEM.CHANNEL.SYNCQ queue. Consistently smaller messages can take advantage of intra-group queuing such that upon receipt of a message destined for a queue on a different queue manager in its queue-sharing group, enabling messages to "hop" to the correct destination via shared queue SYSTEM.QSG.TRANSMIT.QUEUE instead of using a less efficient transmission queue and channel. Transmission queues temporarily store messages destined for a remote queue manager. To send messages directly, at least one transmission queue is defined for each remote queue manager targeted.
This is why early characterization of Message size Avg/Max is helpful in making configuration decisions.
A message queue channel is "in doubt" when the sending message channel agent is waiting for acknowledgement of receipt for a batch of messages being processed. So information needed to resolve in-doubt units of recovery must come from the coordinating system. WebSphere MQ loses its connection, upon restart it attempts to recover all inconsistent objects.
When a shared queue manager detects another queue manager has terminated abnormally (disconnecting from central z/OS Coupling Facility structures), it performs a multi-phase "peer recovery" process
The number of messages which can fit in a queue (a queue's maximum depth) is the combination of the message size and the size of the persistent z/OS page set where object definitions and message data are stored on non-shared queues. As messages are removed from a queue, space in the page set is freed for reuse.
In-flight (not committed) units of work being performed on the failed queue manager are backed out if the Backout Threshold has not been reached. ???
During the first phase, units of work that progressed beyond the in-flight phase are recovered. This involves committing messages for units of work that are in-commit and locking messages for units of work that are in-doubt. During the second phase, uncommitted messages related to in-flight units of work are rolled back, and input-exclusive information about active handles on shared queues in the failed queue manager are reset, allowing other active queue managers to open the shared queue for input. Request messages in a syncpoint (synchronization point when all the recoverable data that an application program accesses is consistent but not yet put to the response message) are put back on to the request queue and become available for another server instance to process.
IBM WebSphere Application Server Network Deployment V6.0 replaces external clustering software like IBM HACMP with a hot peer failover capability for critical services.
MQI CallsApplications using the WMQ API (MQI) issue 13 different calls:
MQ Message DescriptorsMQ messages also contain a Message Descriptor (MQMD) which contains metadata about (properties of) the message and its application data:
Queue Manager CommandsCreate QM:
(-q specifies the default queue manager)
-lc (use circular logging),
-ll (use linear logging),
-lf (specify size of log file)
-lp (specify directory to hold log files) can be passed with the crtmqm command
Delete a queue manager:
Start a queue manager:
If the queue manager is started for the first time, default objects are usually created.
Inquire, define, and alter a queue manager's objects:
this accepts standard input and standard output options
Display attributes of a queue:
Altered attributes using the ALTER command.
Delete queues using the DELETE command.
Types of QueuesMQ products manage queues (resources):
Other topics of functionality:
Distributed Computing with IBM(r) MQSeries (John Wiley & Sons; Bk&CD Rom edition, October 25, 1996) by Leonard Gilman, Richard Schreiber, Len Gilman
Achieving Production Quality Messaging article by Nancy Cox at Network Computing magazine
Other members of IBM's MQSeries family of product provide specialized functionality:
IBM made MQ open for others to integrate with it (the 'glue' that bonds together many disparate platforms and technologies):
IBM's MQWare is for independent software vendors (ISVs) and Internet Service Providers (ISPs) to embed IBM's MQSeries messaging technology in shrink-wrapped Windows NT-based applications. According to a July 1997 news release: The product provides assured, "once only" message delivery between instances of a message-enabled application (e.g., it ensures information is not lost between servers using MQWare to issue "send and forget" messages between instances of their application in an NT Server-based network).
MQWare was the first product to conform to the Business Quality Messaging concept, a set of standard APIs for suppliers to guarantee messaging delivery.
Historical note:Business Quality Messaging specifications ratified September 1997 by the now defunct non-profit BQM Forum formed April 1997 among AT&T, Compaq, IBM, Intel, Microsoft. aimed to "revolutionize" business-to-business trading by offering an inexpensive and universal method of reliably transferring information, such as SAP R/3 data, between trading partners. WARNING: Spammers and virus distributors have taken over websites formerly owned by the Message Oriented Middleware Association (bqm.org and www.moma-inc.org).
Candle's Roma product
Asynchronous messages owned by various resource managers (managing queues in WMQ or in database tables) can be combined into a single unit of work using the XA transaction management interface published by the X/Open Company.
XA defines a syncpoint coordinator computer subsystem that manages units of work. It determines when updates within a unit of work can be committed or backed out.
IBM's MQI protocol
Transaction Processing Facility (TPF)
MSMQ was code-named "Falcon" during its development within Microsoft.
MSMQ offers message queuing functionality such as message delivery, cost-based message routing, and full support for transactions.
Components of Microsoft's Message-Oriented Middleware (MOM) offering include:
MSMQ is integrated with other Windows NT features such as Microsoft Transaction Server (MTS).
It is "agentless". It issues MQ commands to define triggers which notify events such as Channel Started and Channel Stopped, plus Queue Manager events such as
It also monitors performance triggered events:
WMQ vs MSMQMicrosoft commissioned a study by NSTL [dated May 17, 2000] which claims its Message Queuing (MSMQ) product runs "10 times faster than IBM's MQ Series".
Your first name:
Your family name:
Your location (city, country):
Your Email address:
Top of Page