Extensible Integration of the Enterprise and Beyond

Feb. 28, 2000 12:00 AM

One of the most significant challenges that businesses have traditionally faced is the integration of various system components throughout the enterprise. Over the past year, EAI (enterprise application integration) has emerged as a popular approach for integrating systems and gaining a strategic advantage in the marketplace. A corporation that has successfully integrated their internal systems is better positioned to take advantage of the opportunities that emerge following this effort. A financial services firm, for example, could profit from "cross-selling" products and services that in the past may have been tied to monolithic, proprietary systems that were incapable of sharing information. Tool vendors such as NEON, IBM and BEA have enabled businesses to take advantage of EAI using technologies such as message brokering, MOM, CORBA, COM and others.

The purpose of this article is to suggest a broader EAI strategy that leverages the power and flexibility of Extensible Markup Language. XML provides a platform-agnostic, technology-neutral form of structuring messages. By avoiding specific platform or technical messaging requirements, the approach to systems integration becomes more open and thus more extensible.

This article also examines possible approaches for using XML with database middleware, distributed objects, message-oriented middleware, application servers and EDI.

Issues Associated with EAI
Tightly Coupled Designs
One of the potential issues associated with EAI is its limited scope - EAI is concerned with a single enterprise. The success of the business-to-business (B2B) e-commerce model has required firms to focus beyond their internal systems. Businesses interested in automating their supply chain via an extranet or communicating with trading partners via well-defined B2B interfaces are confronted with much more challenging integration issues than the typical EAI project. For example, a company's middleware or messaging product may not be fully compatible with all trading partners. (Promotion and adoption of a B2B strategy is most successful when partners' technical demands are minimized or effectively eliminated.) Since EAI typically focuses on integrating applications within a single company, a data-level or API-level approach is usually chosen.

A data-level approach to EAI focuses on the processes, techniques and technologies required for transporting system data from one data store to another. (The term data store is used here instead of database since some EAI efforts utilize both relational and nonrelational data.) Typical processes may be devoted to extraction, transformation and loading of the data (commonly referred to as the ETL model). Extraction of the data typically requires some form of middleware (such as ODBC), while the transformation and loading of the data can be accomplished via custom software and/or COTS (commercial, off-the-shelf) packages.

An API-level approach to EAI requires the use of programmable interfaces exposed by system vendors that can be utilized to develop third-party extensions and system integration. While this approach to EAI has the advantage of using proven, mature technologies (vendor APIs are usually very reliable), the functionality of the interfaces varies depending on the vendor. The costs associated with hiring and retaining developers familiar with vendor APIs may also be somewhat prohibitive.

While there are several advantages to using a data-level or API-level approach to EAI, the end result is usually a tightly coupled design that can be extremely difficult to extend to partners outside the enterprise. However, if the design requires all messages to be constructed in a platform-neutral format using XML, it becomes much more flexible and significantly lowers the entry barriers for partners outside the enterprise.

Tool Immaturity
As stated earlier, most EAI vendors tend to focus on and promote the technical capabilities of their products (especially in terms of middleware) instead of building expertise in specific business processes and business/application semantics. Eventually they will begin to build expertise in these areas (rather than the one-size-fits-all approach currently favored by many vendors). For now, EAI vendors continue to focus on technical features since they're easily marketed and understood. A more nebulous feature (such as business semantics) can be extremely difficult to market and implement.

The emergence and subsequent popularity of XML helps facilitate the development and integration of business and application semantics - enterprises can define their own data elements (e.g., tags) to better communicate the "meaning" of their data. XML's self-documenting design enables external partners to communicate with a greater level of clarity, ensuring that message constructs will be processed in the appropriate manner.

Message Brokers
Message brokers are frequently referred to when discussing EAI vendors and tools. Unfortunately, their function usually isn't well understood. They behave much like message-oriented middleware products (such as MQSeries) with some additional capabilities. These additional capabilities vary with the associated vendor but usually consist of:

Transformation options

Store and forward messaging

Bridges for communicating with specialized COTS packages

Management tools for tuning the transformation rules engine and monitoring overall performance

A listing of message broker vendors and products appears elsewhere in this article.

While message brokers do a good job of connecting applications via messaging, they don't adequately communicate the semantics of the message being transmitted. As indicated previously, XML should be used to format messages for transmission since it preserves the semantics. Some message brokering tools, such as MQSeries Integrator, already offer limited support for XML (messages can be transformed by the tool's rules engine into XML using an agreed-upon schema).

Business Processes and the Hub/Spoke Architecture
A typical EAI solution employs a hub-and-spoke architecture as illustrated in Figure 1.

Applications within the enterprise access data from a central hub via application spokes. Data within the hub is populated via ETL processes that can be colocated at each application and/or within the hub itself. While this model appears to be ideal, it rarely reflects the nature of the relationships between the associated applications. Many business processes don't easily fit into the model illustrated in Figure 1. In many enterprises business processes can (and do) cross both functional and organizational boundaries, resulting in cross-system dependencies that may exist outside the hub/spoke architecture. The data required to use these overlapping processes may be dependent on the sequence and/or frequency of the ETL processes. Integrating this data can be extremely difficult to manage.

One possible solution to this issue is to ensure that the data stored at the hub uses an open, extensible format that can be quickly and easily accessed by multiple business processes. Leveraging XML for the EAI hub helps ensure that the data is stored in a well-defined (assuming common storage schemas are used), highly accessible format.

XML and Middleware
This article has focused on the potential issues associated with EAI and how XML can be leveraged to minimize these issues. Middleware is the enabling technology for EAI. Implementing an EAI solution is simply not possible without using some sort of middleware.

The remainder of this article discusses how XML can be used to extend existing middleware tools and processes within and across multiple enterprises. It's important to note that, while XML is capable of enabling a great many things, it can't and won't (at this stage) completely replace tried-and-true middleware solutions. It can, however, extend the flexibility and openness of these solutions.

XML and Distributed Objects
The category of distributed objects covers a fairly broad spectrum of middleware solutions. We'll categorize the following ones as distributed objects:

Remote Procedure Calls (RPCs): This approach, while technically not objects in the classic sense, encompasses technologies such as DCE (Distributed Computing Environment).

Common Object Request Broker Architecture (CORBA) (as defined by OMG): Enables applications from different locations and vendors to communicate with one another via an ORB (Object Request Broker) by trapping method calls and mapping them to the appropriate object.

Component Object Model (COM)/Distributed Component Object Model (DCOM): DCOM was developed by Microsoft and is conceptually similar to DCE. Applications developed for the Microsoft environment contain interfaces that enable client objects to request services from one or more server objects regardless of their location on the network. DCOM grew out of Microsoft's COM, which enabled client and server processes to communicate on a single machine.

Enterprise JavaBeans (EJB): EJBs are similar in concept to CORBA but are tied to a single programming language implementation (Java). The EJB architecture enables developers to create robust server-side components (written in Java) that can be reused by clients or other server-side processes.

While XML shouldn't be viewed as a replacement for the above-listed solutions, it can be used to extend their flexibility. Listed below are several initiatives currently underway to use XML with distributed objects. RPC
Transportation of RPC messages has traditionally required the use of "closed" wire protocols such as DCOM or CORBA IIOP. The XML-RPC specification explains how proxies and stubs can communicate via HTTP and use XML as the encoding mechanism (instead of DCE RPC Protocol Data Units [PDUs]), and it's been implemented for several platforms and programming languages.

CORBA and COM/DCOM
The massive growth of Web-based application development prompted the W3C in May 1998 to issue a Note on incorporating the benefits of distributed objects in a platform-neutral manner. The traditional approach of POSTing HTML forms lacks the security, scalability and object interoperability of a typical distributed system. The W3C Note explains how proxies and skeletons can communicate via HTTP, XML and URIs. This approach results in a far simpler implementation since the transport mechanism (HTTP) is already provided. Formatting messages with XML enables an open and flexible standard to be used for marshaling and communicating object metadata for a CORBA interface repository or Microsoft COM/DCOM registries.

EJBs
Sun Microsystems' EJB specification is a cross-platform component architecture for the development and deployment of multitier, distributed, scalable, object-oriented Java applications. Unlike CORBA and COM/DCOM, EJB-based distributed objects are language specific (they can be written only in Java). An EJB-enabled server manages containers that in turn manage one or more EJB classes (or instantiations of classes). The container can be thought of as a "wrapper" that provides additional services (transaction control, lifecycle management and security) to the bean. The EJB deployer installs EJB classes on the server and is responsible for configuring such properties as the location, schema and transactional capabilities of the underlying database.

Sun is planning to use XML for use with EJB deployment options (also known as deployment descriptors). The EJB 1.0 specification relied on a serialized format that was difficult to configure as it was hard to determine who was the information provider and who was the information consumer. As with the previously mentioned distributed object categories, messages (in this case, serialized Java objects) can be formatted with XML to maximize flexibility.

Additional information regarding the EJB specification appears at the end of this article. XML and Message Queuing
The primary products in the Message Oriented Middleware (MOM) arena are IBM's MQSeries and Microsoft's MSMQ.

In June 1999 IBM announced native support for XML within its MQSeries family of products. MQSeries messages can now be formatted into XML and transmitted using the MQSeries transport. An additional MQSeries product, the MQSeries Integrator, provides the ability to bridge between XML-based and non-XML data, thereby accelerating the adoption of XML as a standard messaging format. (It should be noted that messages that are formatted using XML will result in dramatically larger message sizes, potentially impacting message processing performance.) IBM plans to provide further MQSeries support for XML via the Common Messaging Interface (CMI), which provides a logical message construction API similar to the MQI (Message Queue Interface, the MQSeries API). CMI will provide the ability to construct and parse messages regardless of their physical representation and will be capable of parsing both XML and language-dependent structures (such as those found in C, COBOL and Java).

While Microsoft's MSMQ product has yet to provide direct support for XML, messages can still be formatted in XML and transmitted using the MSMQ transport.

Summary
This article has attempted to document possible uses of XML within EAI and B2B integration scenarios, and a common thread running through it is formatting messages using XML. This helps avoid a tightly coupled design that can potentially lock out other applications or companies.

While XML won't immediately replace standard EAI tools and middleware solutions, its self-documenting design and ability to store and communicate metadata help preserve a flexible, open-applications architecture.

Related Links

▪ Figure 1

▪ Figure 2

▪ Figure 3

About John Evdemon
John Evdemon, formerly coeditor-in-chief of XML-Journal, is an Architect with Microsoft's Architecture Strategy Team covering BPM, SOA and Internet Scale Computing. He is an XML and e-business expert, having served as CTO/Director of XML-Related Products for both a large integration platform vendor and a small XML-centric start-up. He has been working with XML since its early beginnings, is an Invited Expert with the W3C XML Core Syntax Working Group and has chaired several industry-specific XML initiatives.