The holy grail of e-business and Internet technologies is the ability to enable cost-effective business-to-business transactions with trading partners eliminating the inefficiencies of paper trails, duplicate data processing, and rekeying of information. Companies are increasingly calling on IT departments to develop a network of relationships encompassing suppliers, business partners, and internal activities that is accessible, integrated, scalable, and resilient.

To stay competitive, companies must be able to transform and exchange business data freely and dynamically between their own systems and those of their business partners via the Internet. This involves the integration of many different corporate databases, applications, Web sites, and business partners.

Web services promise to provide a way to more easily and cost-effectively exchange, transform, and access corporate data on many different incompatible systems, both within and beyond the walls of the organization. The Web services concept uses the Internet and an open set of standards based on XML as the common data interchange format.

But before B2B or Web services can be considered, an organization may have to deal with significant internal challenges to ensure the interoperability of various systems and applications as well as the integration of various sources of critical business data. Corporate entities generate vast amounts of data every day - transactions, inventory, product information, order status, sales and customer data, invoices, payments, business contacts, reports, statistical data, Web access logs, and so on. In most cases data is stored and managed by various database systems. As businesses grow over time or rapidly evolve through mergers and acquisitions, they acquire many heterogeneous systems that may not easily communicate with one another.

As companies try to steer a course through the hype and develop strategies for implementing B2B and Web services, they will face the challenge of integrating these existing relational databases, EDI (electronic data interchange) infrastructures, and text formats with XML. Real-time XML-driven middleware solutions will play an increasingly important role in helping companies integrate their existing data assets with XML to enable effective B2B relationships.

B2B Hurdles
For B2B to realize its potential, transactions between business partners must be conducted electronically in an interactive manner to eliminate tedious manual processes such as rekeying information. The challenge is that no two companies have the same database structure or format their data in the same way. The ways in which we store and process data are often so diverse that either:

1.   The business partner would have to deliver the data in many different formats OR

2.   The subscribers or recipients of the data would have to transform the data in order to use it effectively OR

3.   They'd have to adopt the same systems as the data publisher and a universal standard for data exchange.

The conventional method of electronically exchanging data is to write custom programs to generate and read data in a format agreed to by two parties or a group of participants. When one partner needs to change their database or data structure, either the company will have to rewrite programs to meet the agreed-upon standards, or all other partners who exchange data with this company will be forced to make changes to their own programs accordingly. Adapting to ongoing change results in very high maintenance costs. Companies also have to maintain skilled development personnel in-house or outsource development.

Some large enterprises have implemented sophisticated EDI solutions through a specialized value-added network (VAN). The traditional EDI infrastructure is not only extremely inflexible and expensive to implement, but can also result in high maintenance costs when adapting to ongoing business changes. For small to medium-sized businesses, EDI is often beyond reach. Another major problem with traditional EDI is that it requires a unique solution for each pair of trading partners, resulting in a high transaction cost.

XML presents a more efficient solution for data exchange through value-added transformation and by leveraging the Internet or internal networks connected to enterprise databases in a secure environment. Rather than force data subscribers or receivers to adapt or transform the data format the sender chooses, the data publisher or sender can deliver data in XML that the subscriber or business partner can easily retrieve, view, and use for whatever purpose.

Strengths of XML
Using XML, virtually any data items can be identified, allowing Web pages to function like database records and paving the way for a truly transactional Internet. XML doesn't just relay the data that needs to be transported, it contains information, or metadata, about the data that it's sending. This means that the data is absolutely self-contained within one package with no need to agree on columns, rows, or other data formats or syntax. By providing a common method for identifying data, XML effectively supports B2B transactions. Some of the key features of XML are summarized below.

Extensibility
Using XML, information publishers can define new tags and attribute names at will. Rather than being constrained to defining a particular set of data, XML is able to work with DTDs and schemas to define any number of documents that form a language of their own. Hundreds of thousands of specific document vocabularies have already been created using XML to meet the specific needs of various industries.

Structure
XML was designed for maximum expressive power and is applicable to almost any data or document structure no matter how complex. With XML, document structures can be nested to virtually any level of complexity, allowing the representation of database and object-oriented hierarchies.

Validation
XML documents come with built-in error and validity checking. They may contain a description of its grammar for use by applications that need to perform validation of the information contained in the document before populating a database or executing a transaction. For example, the DTD or schema can be used to verify that all elements are correctly specified, in the right order, and that values fall within acceptable predetermined ranges.

Loosely coupled architecture
XML merely represents information and metadata about the information without indicating any specific way in which the data should be processed or putting any limitations on how the information may be handled or displayed. Because XML separates data from processing, it offers huge benefits for companies seeking a multiplatform, multidatabase solution for B2B.

XML documents can be freely exchanged across multiple platforms, databases, and applications so long as the subscriber data stores and applications are XML-aware. Any system, mobile device, or application that speaks XML can access and manipulate the data contained in the XML document at will. This makes XML ideal for enabling B2B where companies often face the challenges of mixed-system environments. Systems built on a loosely coupled XML integration framework are future-proof because changes to the computing environment won't affect XML data exchange.

The unique capabilities of XML, including vendor and system independence, make it an ideal enabling technology for applications that require data to be communicated between two or more heterogeneous databases within the same company or beyond the walls of the organization to include business partners or suppliers.

Challenges of XML Adoption
Since its introduction, few technologies have been hyped as much as XML. But while XML is seen by many as a key B2B enabler, it's really only the first step. XML tags provide a simple data format, but the intelligent defining of these tags and common adherence to their standard usage will determine the real long-term business value of XML.

Already there are several different XML schemas being promoted by different industry coalitions or bodies, including W3C, OASIS, and BizTalk, as well as Web sites that provide repositories (Biztalk.org and XML.org) for publishing and reviewing XML schemas. Hundreds of XML standards have been created since the introduction of XML, which raises the question of whether or not a standard for XML actually exists. XML standards are currently in flux. The potential for the creation of many different "proprietary" XML documents threatens to undermine the successful adoption of a true XML standard and hinder the widespread deployment of XML solutions across the enterprise.

Role of Data Integration Middleware
The successful adoption of XML for data transformation, exchange, and integration depends on the support of leading integration middleware vendors who can provide out-of-the-box solutions for integrating existing or legacy systems that may not be XML-aware, and ensuring that users can easily and cost-effectively transform data between other data formats and XML as well as between different competing XML standards should this be necessary.

The need for a certain amount of transformation between document formats is inevitable. XML-based integration middleware is necessary to enable users to freely and seamlessly flow and transform data between text, database, and XML. This technology ensures that XML-aware and non-XML-aware systems can coexist, enabling companies to realize the full potential for XML data exchange and integration. XML-powered integration middleware has a role to play in brokering B2B data transactions between legacy applications and application server technologies to enable companies to tightly integrate all their systems into a cohesive infrastructure without requiring custom development or changes at the application layer.

Enterprise data is typically structured as repeating sets of hierarchical entities such as those stored in a relational database. Traditional transformation engines often scale up very well but lack flexibility and cannot deal with hierarchical data. Emerging XML middleware technologies provide an excellent vehicle for developing a new generation of scalable and flexible transformation engines.

XML-powered data integration middleware plays a key role in enabling effective B2B communications by bridging the heterogeneous databases, EDI infrastructures, and text formats deployed by your organization and by your business partners (see Figure 1). XML middleware adds value to B2B implementations by allowing you to leverage your existing mix of systems, including technologies that may not be XML-aware.

For XML-based transformation engines, both DOM and SAX models have substantial limitations in handling enterprise data, scalability, and flexibility. The most advanced middleware technologies take the advantages of both SAX and DOM models and intelligently apply them globally or locally, wherever they're required, to best achieve the desired results. This processing model not only provides the best performance and scalability but also maximizes flexibility.

Choosing a Middleware Solution: Selection Criteria
The middleware market is growing increasingly cluttered with multiple vendors, solutions, and approaches vying for your attention. How can your business see through the clutter and choose an effective middleware solution to enable your B2B initiatives today and in the future?

The following considerations can be used to guide you in your search for an effective middleware solution to connect your internal systems and data sources, integrate your data assets with XML, and pave the way for successful B2B and Web services projects.

Bidirectional, multidatabase data transformation
In a business-to-business environment, data interchange is the norm. You receive XML documents from your business partners or another business unit within your own organization and you want to automatically save the data from those XML documents to your database with your special requirements. The middleware tool should have the capability to transform and flow data bidirectionally between database, XML, EDI, and text formats in any combination to leverage all information across your enterprise. Through XML, data can be transformed from any format to any other format for powerful distributed data applications. XML documents created by the middleware solution should contain not only data but also the relationships of the data. This lets you map the objects between your XML document and database with value-added transformation rules, and update your database accordingly.

Hierarchical data transformation and hierarchy reconstruction
The tool should allow you to transform hierarchical data between database and XML, database and database, XML and XML. The tool should read and map databases to and from XML according to their hierarchical relationships. It should maintain relationships of data by reconstructing the hierarchy and mirroring the relationships in the original database. Look for a middleware solution that can automatically format a flat query result into a hierarchical structure, allowing administrators to easily configure and map data transformations.

Customizable rules and data manipulation
For maximum flexibility the middleware tool should be able to manipulate the data itself as well as the data structure or format to generate unique key values required by the database. Administrators should have the ability to incrementally update databases, add extra fields required by the target, assign fixed or dynamic default values to fields, completely change data structures, and apply formatting rules to data.

Support for XPath expressions and functions
XPath expressions and functions are used to map the data source for cross-level object mapping as well as mixed mapping between element and attribute. The middleware solution should support built-in XPath and complex expression mappings, as well as complementary functions to XPath and XSLT such as date formatting, string conversion, and obtaining system date and time. It also should accommodate database-specific extensions for stored procedure calls, data lookup, and generating key values using the database's stored procedures and sequences as well as Java objects.

Open architecture
Open architecture enables users to plug in Java objects for complex processing requirements, such as data validation and translation with lookup tables. Open API class libraries and documentation enable third-party applications to easily interact with the middleware solution.

Built-in query capabilities
To transform data and generate reports, companies have typically written custom-coded programs. These programs can't be easily shared or reused by different applications or systems. This means that enterprises have to write and maintain a program for each usage scenario and for any change in the computing environment. Custom programs developed in-house lack the ability to keep pace with the changing needs of enterprise systems. Developers have to constantly rewrite the report generator program to adapt to changes. The result is a very high and ongoing maintenance cost. Another issue enterprises often encounter in the generation of reports is that complex queries can cause considerable overhead on server resources, significantly slowing down key business processes and, in the worst-case scenario, resulting in system downtime. To overcome these potential problems, some companies summarize the data at off-peak times after normal business hours. While this batch process resolves the resource problem for production servers, it creates a new problem: the data used to generate reports is stale and out of date. The middleware solution should have built-in support for running effective queries, report generation, and real-time analytics.

Summing Up
Wise investments in XML, as well as XML-powered integration middleware, can effectively future-proof your business against many of the pitfalls of changing information systems, best-of-breed applications, and multiplatform computing environments. Moreover, XML promises to provide a powerful tool for building bridges between your organization and your business partners and suppliers that will help you achieve significant business returns. The next generation of XML solutions will enable true real-time data exchange between diverse systems, both in-house and beyond, to encompass business partners and suppliers.

An effective XML-based B2B strategy can help your business realize new operational efficiencies while enhancing service levels and maximizing partnership and revenue opportunities. New middleware tools that support open standards like XML are well worth evaluating. Thorough testing and proof-of-concept will determine whether these technologies are a good fit with your organization's data infrastructure.