Leveraging Legacy Assets

Given sufficient budgets, of course, we could all recommend one of several strategies for modernizing legacy applications.  Buying replacements off the shelf or rewriting/redesigning each application in J2EE, .NET or BPM technologies would head the list for most CIOs.  In general, agile methodologies are proving that they can reduce the risks of new development strategies to acceptable levels in most cases, provided the scale of the project is not too large. 

However, what if you have neither the budgets nor the time horizons required to implement a replacement strategy?  Or, if your previous experience has been that replacement projects may fail or experience substantial cost and/or delivery overruns? Or, if the project is too large to apply agile methodologies?  Then what options are available to you?

Example Assessment Study of Client Portfolio

Assessing Residual Value

At first glance, the portfolio in our study looked like a mess. We had on the order of 10,000 programs ranging in age from 10 to 30 years. Many of the programs were very costly to maintain due to constant maintenance modifications over the years. Very little of the data is stored in a relational database management system. Several critical applications do not even run under any kind of transactional control, including applications running under CICS which would normally provide transactional control. Restart/recovery consists of reloading files from tape and re-entering transactions.  All in all, surveying these programs was like stepping in a time machine and coming out in 1980.

When the application owners were interviewed about how well these applications cover the present business purpose, we heard a familiar litany of shortcomings in the applications. They wanted a graphical user interface rather than character based "green screens". They wanted the data stored in a relational database where it could be queried on an ad hoc basis. They wanted reliable restart/recoverability.

When we drilled down with each of them, however, we found that the existing applications provide support for the business purpose well in excess of 90% for all of them, and almost 100% for several. When they were saying that they wanted “modern” applications built on J2EE, .NET and BPM, they were confusing the infrastructure of relational databases, application servers, and graphical user interfaces with the logical expression of the application business rules.

This is the key question that we ask during a portfolio assessment: does the legacy application serve the existing business process adequately, or has the business process changed significantly since the application was designed and implemented? If the application serves the business process well, then it is a candidate for a legacy modernization strategy as a complete solution, or for a modernization strategy coupled with subsequent application modifications. 

However, if the application no longer fits the business process well at all, then there is little to be done except perhaps extract the data model as a start toward building a new application.

Application Replacement Risks

We also discovered from the application assessment that the program source codes contained business rules that were no longer reliably part of the institutional memory. Many key people instrumental in their design and implementation had retired or otherwise left the organization, and of course the documentation is far from current, if indeed it was ever correct. Attempting to replicate support of the existing business process therefore had a significant risk of error, if it were elected to replace the systems with newly written applications. The only fully accurate expression of the business rules exist in the legacy source code itself.

This constitutes a huge risk factor, and one that would be extremely easy to overlook at the beginning of a replacement project, particularly for technicians not trained in software archeology. This is the classic situation in which a rewrite/replacement project can get to a 95% completion status on schedule, and then stall there for months or even years while programmers tinker with the business rules, slowly getting them right while budgets and careers take it in the neck. The poster child for this was the bank in Los Angeles which, when we arrived for the assessment, revealed that they were 10 years into an 18 month project - with no realistic end date in sight. During such a nightmare, the costs mount while the benefits recede into the future.

This risk assessment must be a major consideration for whatever solution is recommended . Indeed, controlling risk was one of two primary drivers of the final recommendations of the study, cost being the other.

Note also that for this specific situation, a rewrite approach using agile methods offers little or no risk improvement, because the test first paradigm that is central to agile cannot be utilized effectively. Test first implicitly assumes that the focus is on getting the coding right to meet a well understood implementation requirement. This situation turns that otherwise excellent paradigm on its head: we can't tell if the coding is right, because we don't know that the design specification is itself correct until we test the coding against the business process in real time.

No feature of agile is prepared to deal with replacing a functioning system that no one understands at a sufficient level of detail to derive accurate specifications. Some form of data mining of the old code would be required, at a minimum, and this will probably need to be augmented with detailed research among the day to day users and perhaps even with appeals to retirees.

Finally, of course, we have the budget constraints. Rewriting 10,000 programs, even if we could replace half of them with a query tool, would consume on the order of 10-20 years worth of capital budgets for this organization. This is simply unrealistic, and even if we had the money we don’t have the time. Spending a lot of money now for benefits so many years in the future would never secure management approval.

Modernization Recommendations

Leaving aside the one application system that we did recommend for replacement, because it covered the current business process less than 50%, we placed the assessed code into one of two categories:

• 3GL code (principally COBOL) running in different mainframe and open systems environments.  We recommended that -
  • this code should be retrofitted with a relational database,
  • the online transactions should be brought under a transaction processing monitor with COMMIT processing and decoupled from the user interface presentation with an XML parser,
  • the spaghetti code should be restructured to ease the maintenance burden,
  • an external portal server should utilize the XML interface onto each platform to provide a GUI for end users, and
  • a BPM engine should access the transactions directly via XML (or web services) to facilitate automating and re-engineering the business process.
     
• 4GL code (principally Informix and Natural) running on Unix and IBM mainframe MVS.  We recommended that this code be re-architected into a modern platform, either .NET or J2EE, by data mining the business rules in the source code.

We estimated that the 3GL code renovation could produce all of the desired new capabilities including significant maintenance cost reduction in less than 20% of the cost and time of a rewrite that actually delivered on time and on budget.  Unfortunately, of course, as we saw in the new Capers Jones study, the chance of that happening was less than 10%.  The re-architecting of the 4GL code was expected to be closer to 50% of the cost of a rewrite.

The reality is that the only practical options besides modernization would be buying a package (which has its own peculiar risk factors) or outsourcing the application to a service.  Rewriting would simply be impractical.

Summary

It would be reasonably unusual for a commercial site to have as heterogeneous as environment as this.  In this one study, we covered what we might see in 5 or even more standard commercial IT shops.  However, you may find your site among one of these descriptions.  All of the alternatives discussed are technically reasonable, and some are even inexpensive.  They are all less expensive, usually much less expensive, than a conventional redesign and rewrite approach, and much less risky. 

A COTS package (or an outsourced service which amounts to the same thing) that can be used as is or with configuration changes can be a viable alternative, though none was applicable in this study.  However, if more than non-trivial changes to a package are required, we would recommend proceeding very cautiously with that strategy.

There is a great deal more information in the referenced article, and if there is a chance that you would consider a modernization strategy for your needs then we recommend that you read it, our new essay "It's Not Magic," or both.

If you wish to consider a similar assessment for your legacy portfolio, please contact us.