Industry research shows that while many companies believe SOA is beneficial, only a few have reached a level of maturity that lets them to reap real benefits. The relative immaturity of SOA across IT organizations, however, presents an opportunity to structure the SOA program in the most effective way. If not positioned correctly from the very beginning, SOA may not show any benefits and will eventually be written off as yet another failed initiative. Creating the most effective structure for the SOA program, establishing a clear and actionable roadmap for the future, and clearly demonstrating the accomplishments will ensure success of SOA across the enterprise.

A SOA program cannot survive too long purely as an IT initiative. Eventually it needs to show the return-on-investment (ROI) impact on the company's bottom line and business benefits. To do that, a well-defined set of metrics has to be established from the very onset of the SOA program, data has to be collected on continual basis and information disseminated to all the stakeholders. Not considering ROI in the initial stages of the SOA program can lead to an inability to show real benefits or progress. Thus, financial and business meaningful metrics should be considered essential in the success of SOA in any organization.

Defining the Metrics
Metrics are critical to the SOA success. Without defining a clear set of measurements, progress and maturity of the SOA program can not be assessed. Thus, metrics relevant to the organization's SOA roadmap and maturity model should be defined as early as possible. All of the processes and approaches discussed below are based on the assumption that metrics can be effectively captured and communicated.

There are typically three types of SOA metrics - IT, business, and financial. Each represents a view relevant to a specific group and is focused on maximizing their understanding of how SOA impacts them. Table 1 contains a sample of each metric type.

Each organization has to decide which metrics are most relevant in capturing SOA value, demonstrating progress, and influencing appropriate outcomes. These metrics may vary across the companies. However, their goals should remain the same. When defining the metrics relevant to your organization, think about the goals they would help to achieve and choose the ones that would help move the SOA program as well as the whole company forward in the most effective way.

Collecting the Metrics
Once the metrics are defined, a detailed plan can be devised to capture them. The key is to set up an easy and repeatable process since it will have to be executed numerous times. Also keep in mind that the data being collected will be used not only for metrics calculation but for reporting purposes. Thus, setting up a capture and storage mechanism with both of these goals in mind will help minimize process changes and future efforts.

Regardless of how the services are being delivered and who creates them, there are several key pieces of information that almost always need to be captured.

1. All the services being created
2. Cost to build each service
3. Integration costs related to service reuse
4. All service reuse opportunities

Depending on the actual metric, additional information may have to be collected. However, the four data points identified above will most likely meet 90% of your reporting needs.

There are numerous ways to collect the raw data needed for SOA metrics creation. Some service management products can keep track of the number of services calls being made, who calls each service, and other service performance information. Registry/Repository (RegRep) tools can help keeping track of how many services exist and how many are under development. General time reporting software can be leveraged to determine the time and associated cost related to service design, development, testing, refactoring, and integration. The real trick to doing this effectively is to set up the appropriate reporting buckets so that individuals working on a variety of service delivery tasks can charge their time appropriately. ETL (Extract-Transform-Load) tools can be used to retrieve data from a variety of data sources and bring it all together in a single location. Figure 1 depicts the automated metrics collection process.

Of course, a manual process for keeping track of all this information can always be established. However, relying on the existing software products to produce the necessary information represents a more reliable and transparent approach since it provides a greater validity to the end results.

Calculating ROI
Wikipedia defines ROI as the ratio of money gained or lost on an investment relative to the amount of money invested. Calculating the ROI for the SOA program should follow this definition. Holistically, the SOA program ROI can be calculated as follows:

SOA ROI ($) = Cost Savings/Efficiencies Achieved - All SOA-Related Investments

or

SOA ROI (%) = SOA ROI ($) All SOA-Related Investments

It's generally easy to determine what the total SOA-related investment looks like. This amount should include all the costs related to building the existing set of services plus the SOA software purchases such as ESB, RegRep, service management tools, etc. Unless the shared infrastructure and software investments are recouped through other means, they should be included into the SOA ROI calculation.

Cost savings or efficiencies gained as the result of the SOA program are a little harder to define. Efficiencies can be shown as time and cost savings associated with faster project deliveries when reusing services. This, however, implies that overall cost and effort across similar projects using SOA and those that do not must be compared. Alternatively, a baseline would have to be created to define how long the same project would take if it did not use existing services. These are tough metrics to collect and typically require extra effort. So not many organizations choose to track them.

The most popular SOA metric demonstrating financial results is cost avoidance. Projects that leverage existing services are assumed to avoid costs they would otherwise have to incur. Thus to calculate total savings related to SOA, all cost avoidance numbers associated with all documented instances of reuse would need to be added together. Plugging that value into our ROI equation would yield the final result.

Calculating cost avoidance can also prove to be tricky. The easiest way to determine cost avoidance is on the project-by-project basis. The basic formula for calculating individual service cost avoidance as related to a specific project is provided below.

Service Cost Avoidance = Service Build Cost - Project's Service Integration Cost

Where

Service Build Cost = Initial Service Build Cost + Cost of all Subsequent Changes

To calculate the entire project's cost avoidance amount, simply add the cost avoidance for all the services being leveraged. To forecast the total potential cost avoidance at any point of time, multiply the number of times each service is envisioned to be leveraged by its build cost and add it all together. Since the integration cost for each ongoing or future project can only be estimated, a standard reuse factor can be applied to the service build cost. Eighty percent is the typical number used in these situations. Note that projects creating services should not count towards cost avoidance since no costs are actually being avoided.

Publishing the Metrics
Once metrics are collected, they need to be distributed to all the SOA program stakeholders. Depending on the organization, it can be the IT managers and executives, business executives, and partners. It is imperative to the success of the SOA program that clear and transparent communication takes place to demonstrate the progress made and value created. Business and IT executives sponsoring the program need to understand how the funds are being spent and what benefits they help to achieve. It is also important that metrics are verifiable and are based on trustworthy data sources. This ensures transparency and adds validity to the information being communicated.

Metrics need to be published on a regular basis. Determine the best period of time - monthly, quarterly, annually - to provide regular updates and set the expectations. Establish a consistent format and try not to vary it too frequently, so your audience knows exactly what is being communicated and how.

Establishing a centralized SOA metrics dashboard or portal is important. Anyone - from the SOA program stakeholders to the developers - can view the metrics and assess the program's progress, SOA adoption rate, and overall maturity. Sophisticated reporting tools can be employed to allow users to drill into specific metrics and understand the underlying data. The ideal future state for SOA metrics reporting should incorporate sophisticated reporting tools, automated real-time data collection, executive dashboards, and a maturity meter.

Increasing SOA Adoption
Once the SOA metrics are collected and published, the ROI calculated and the SOA program progress tracked, how do you use it all to increase SOA adoption? The answer is simple - create specific, attainable service reuse and creation goals for each IT group and measure them against it!

Service reuse is not going to happen by itself. Opportunities have to be identified, researched, validated, and realized. Everyone in the IT organization - from the executives all the way down to the developers - has to be committed to reuse and make it a priority. Without a strong incentive to do so, reuse will become accidental. The whole organization needs to look for service reuse and creation opportunities at every step - projects, internal efforts, migrations, upgrades, service enablement of legacy systems, etc.

The best way to incentivize service reuse and creation is to establish formal goals by which each IT group will be evaluated at the end of the year. The goals must be attached to the performance objectives of each group's leadership team. The end-of-year evaluations must be meaningful. Inability to attain service reuse and creation goals must be reflected in the overall performance evaluation not only for the IT group but for the whole leadership team. Negative results could yield budget reductions for the IT groups or similar tangible impacts. For the IT managers, inability to meet service reuse and creation targets should result in less favorable performance reviews. Alternatively, meeting or exceeding the goals can result in increases in budget and staff, bonuses, promotions, special recognition, etc. The bottom line is that the incentives should be developed and enforced in such a way that IT groups and their leadership teams strive to reach the established goals and not meeting them brings tangible negative effects. Figure 2 illustrates the process to tie the metrics, objectives, and performance evaluations together.

The formal goals should come down from the CIO and the entire IT senior management team. The whole IT organization must understand the importance of reuse and develop a strong desire to attain it. A strong support of this direction by the IT leadership in the form of formal objectives should influence everyone's behavior. However, without a system of tangible rewards and penalties, the approach cannot be fully effective.

Conclusion
The SOA program is not just about the technology and services being created. It needs to show value. Without this, the investment required to grow and sustain the program cannot be justified. IT and business need to understand what ROI the SOA program provides and how it impacts the bottom line. A successful program can demonstrate its value at any point of time via a set of metrics and reports. It will also have the right policies in place to increase SOA adoption across the organization. All of this breeds success and positive ROI. Metrics describe where the organization is in its SOA maturity, ROI depicts how this relates to the company's bottom line and the policies to increase SOA adoption improve SOA value. Together, these processes will make SOA real for any organization.