nd14-article5-lead

Mining Hidden Gems Extract Information Systems’ Value


To print a PDF copy of this article, click here.

Authors: John Kruse and Maura Slattery

More than 10 years ago, the Department of Defense (DoD) Chief Information Officer took a bold step toward broad information sharing by publishing the seminal Net-Centric Data Strategy. Since then, the Services have made great strides by creating many new data sources across the DoD. Still, taking advantage of all this pent-up capability and value remains a difficult task for most of the enterprise.

The data or capabilities within any given program of record (PoR) system may be valuable to others, both known and unanticipated, but often there is little understanding of how we might extract this value or how mining our existing resources might change the way we do business.
This value often can be exposed quickly and at low cost. Nevertheless, Enterprise Integration (EI), the activity that stitches together disparate systems and data, is not well understood or utilized as often as might be warranted. Some of this is because of systemic issues within DoD acquisition, but much of it is due to a perception that EI is big, expensive and high-risk. In short, there is very little recognition within PoRs that the rewards of EI can outweigh its costs and risks. This article outlines how the Air Force’s C2 Constellation program found a ­successful approach to EI by carefully selecting initiatives that are aligned with PoR plans and that are supported by warfighters.

C2 Constellation

Since 2001, the Command and Control Constellation (C2 Constellation) program has been the “sole Air Force program for defining, developing and assessing integration of global, theater and tactical level Air Force air, space and cyber C2 capabilities in support of the joint warfighter.” Until four years ago, the program tried to span EI. It focused on creating enterprise architectures (EAs) to help “Big Air Force” drive acquisition and systems engineering decisions while attempting to effect specific changes with focused EI projects. An underlying assumption behind the program’s top-down efforts was that if someone could simply identify and document smart choices for systems engineering in support of EI, programs could adopt these suggestions and the enterprise would benefit. Over time, however, it became clear that trying to promote EI from the top wasn’t having the anticipated impact but that smaller, more focused, efforts seemed to get better traction.

Over time, however, it became clear that trying to promote EI from the top wasn’t having the anticipated impact but that smaller, more focused efforts seemed to get better traction.

Why Top Down is Difficult

Creating EAs makes a lot of sense. Rather than have PoRs building systems haphazardly with only their own immediate requirements in mind, we should seek ways to standardize and create rational, repeatable patterns that can provide efficiencies in development, integration and operation. However, in order to provide real value and efficiency, EAs need to achieve a critical mass of adoption, and in our current acquisition environment it is difficult to achieve this across a broad and heterogeneous enterprise.

“To be” EAs are by definition top-down and conceptual in nature. To provide value, they require that (1) an acceptable standard architecture can be accurately defined, and (2) that once defined, we can realistically propagate the architecture among the PoRs to realize its benefits. Even when we achieve the first requirement, our decentralized acquisition system makes it very difficult to achieve the second.

EAs may fail because they are poorly conceived, but far more often they fall prey to an acquisition environment that does not reward cross-PoR cooperation and standardization. PoRs are funded, incentivized and judged by how they deliver capabilities in response to a specific set of requirements for a specific set of warfighters. If a PoR fails to provide benefit to its core set of users, the program is by definition a failure. Thus, conforming to enterprise-level architectures or standards that address the needs of a broader community often is reasonably met with, “We don’t have a requirement for that.”

C2 Constellation faced such a situation in which the programs and portfolios with which it worked were not, for a number of reasons, willing to implement the developed EAs. As a result, C2 Constellation’s leadership decided to revisit its broad-front EI approach. Rather than pursue a strategy that emphasized top-down efforts, the program shifted focus to building bottom-up integration bridges among those who were keen to achieve particular tactical ends. These changes could in turn be leveraged to help the broader enterprise. Thus, by helping PoRs meet their specific, documented requirements faster and at lower cost, the whole enterprise could benefit. Since it has shifted its emphasis, C2 Constellation has enjoyed greater impact with PoRs, and for surprising and simple reasons that might have implications for broader information technology (IT) acquisition.

Factors Influencing Success

Any discussion about IT program or project success would need to acknowledge that a wide range of factors influence success and that there are many potential pitfalls from the genesis of an idea to successful transition. In our experience, however, beyond the standard concerns of performance, cost and schedule, EI issues can generally be simplified into three interrelated classes involving risk. The first two have to do with a capability’s alignment with the two major stakeholders—the warfighters and the PoR—throughout the EI effort. The third, limited complexity, involves the ability of the PoR to limit risk through timely delivery of effective capabilities, given complicated technical and operational landscapes. The following are brief explanations of each of the three and how C2 Constellation realized that they come to influence success.

Operational Community Commitment

The No. 1 question we must address when pursuing EI is, “Who is asking for this?” All the varied stakeholders should have a say in acquisition decisions, but the warfighters should take priority. Without their backing, transition may be technically achievable but may never attain its intended ends. This is especially common when initiatives cross system or organizational boundaries, as is common in EI.

Figure 1. Top-Down vs. Bottom-Up Enterprise Integration

nd14-article5-figure-1

There are good reasons why warfighters tend to be eager to experiment with new ideas but are much more selective about what actually moves forward to transition. They are best placed to imagine the ripple effects and potential risks in everything from training to sustainment that comes with a new technology. Additionally, there are significant barriers to any technology that changes the way the organization operates. In the words of Rear Adm. Tom Zelibor, the Navy’s fleet commander during Operation Enduring Freedom and a technology pioneer, “I’ve always maintained that the hardest part of this isn’t the technology, it’s the culture.” Technologists and program managers may understand many things, but we are not the people who can make accurate calls about how much change a command is willing to assume or the true net worth of a new capability within a greater operational context.

Technologists and program managers may understand many things, but we are not the people who can make accurate calls about how much change a command is willing to assume.

PoR Alignment

The second place where we see new innovations and initiatives fail is in their simple nonalignment with the PoRs in terms of established technical architectures, functionality, acquisition strategy or timing for a smooth technology transition. Expecting them to make even seemingly simple accommodations in transitioning capabilities is often unrealistic within the cost, schedule and performance constraints of the program.

In part because they are cross-organizational, bringing an EI innovation or initiative to fruition in the field is akin to running a gantlet where any single issue might stop an initiative in its tracks or sap its ability to get over the next hurdle. Often, these issues have nothing to do with the wishes of the warfighters, the developers or the participating PoRs. For instance, one common problem in transitioning EI innovations is that of cycle-time mismatches in which a PoR is interested but is simply not ready for the innovation as it already has committed its time and resources. Delay may be possible, but frequently the developers and other PoRs must move on to new work, which often involves disbanding the effort. In such situations, it is difficult to revive stalled initiatives—and, when momentum is lost, even great ideas tend to wither.

Limited Complexity

Once we have moved beyond the organizational and social needs for warfighter commitment and PoR alignment, we must deal with the elusive problem of limiting complexity. Under conditions of great complexity, our abilities to understand systems, extract good requirements and develop compelling capabilities begin to fail. Heightened complexity often leads to either analysis paralysis—in which we are unable to decide what to do—or slow and difficult acquisition that misses the mark and underwhelms the end users.

Moreover, highly complex EI initiatives can increase downstream risks as they have implications for acceptance, security, training and maintenance. Typically, the relationship between system complexity and technical difficulty is not linear—that is, as complexity increases, the associated technical difficulties and risks compound even faster. Thus, a complex EI solution can either be difficult to transition or it may be limited in operational use.

Recipe for Success

Figure 2. Enterprise Integration Factors

nd14-article5-figure-2

When C2 Constellation changed its approach to EI, the program was simply trying to find commonsense ways to identify valuable opportunities, develop them and then transition improvements to the field. The program decided to work directly with interested PoRs to find targeted EI solutions and then provide relatively modest funding to perform the work and some engineering and project management support to help the process along.

As a result of our own particular environment, and previous experiences, C2 Constellation’s leadership explicitly set several criteria for selecting new EI projects that were intended to maximize the chances for successful development and transition to PoRs. Every initiative had to be submitted within a focus area as defined by our sister organization, the Air Force Command and Control Integration Center (AFC2IC). Because the focus areas could change from year to year, all projects’ proposals would need to produce a valuable product at the end of each fiscal year, as specific focus areas might not be continued. Beyond this, projects were specifically evaluated in terms of (1) warfighter impact, (2) transition likelihood and (3) cost.

In retrospect, it is easy to see that our measure of warfighter impact stood in relatively well as a measure of warfighter commitment. Typically, if a given initiative was expected to have high end-user value, the war­fighters would show commitment and even enthusiasm. But as mentioned previously, this support was based on their holistic evaluation of the pros and cons of actually using the EI innovation.

Similarly, our transition likelihood assessment was a reasonable metric for the many facets of PoR alignment. By asking the PoRs for an opinion on this likelihood, we were getting their opinion on how well the initiative was aligned with their current and planned architectures and states. They simply did not want to invest time or resources in any effort that was unlikely to help them deliver capabilities to the warfighter.

In retrospect, we found we had been limiting initiative complexity with our one-year focus and our limited budgets. Everyone understood that cost and schedule were effectively fixed and, if we could not produce something valuable within these constraints, the effort would never be extended—much less transition to a PoR. This tended to lower the tolerance for risk and consequently limited complexity as the stakeholders wanted crisp, understandable and achievable initiatives. Additionally, modest initiatives are less likely to violate organizational culture and norms, which can help gain acceptance and successful transition.

A telling example of this approach would be the Integrated Tactical Airspace (ITA) initiative that sought to knit together Army and Air Force tactical systems to dynamically share airspace data in support of more Agile and coordinated operations. This collaborative effort involved three Army and one Air Force systems sharing airspaces through a community-defined data standard.

The initiative had PoR alignment that was cemented by resource sharing among the joint participants. Both the Army and Air Force users were committed to the initiative as they were anxious to finally have a capability that could support the operational vision that had been established.

Finally, the technical complexity of the effort was controlled through the use of the common data standard and a modest, modular development approach. As a result, the developed prototypes are being moved into the baselines of the respective PoRs.

The Bottom Line

nd14-article5-secondaryThe new bottom-up EI approach has greatly improved the effectiveness of C2 Constellation and the value proposition that we offer to the PoRs and the warfighters. Even in cases where a direct transition to the warfighter was unachievable, it was often possible to affect the PoRs positively through new/changed requirements, improved data schemas, etc. In a recent study of initiative outcomes over the last three years, we found that 16 of 19 (or 84 percent) of our speculative initiatives bore fruit.

A positive secondary effect of the new EI approach was the emergence of resource pooling to achieve results. PoRs are willing to contribute substantial time and complementary resources, and this contribution then cements a high level of commitment to the team effort. The warfighters, in turn, have been positive about collaborating on crosscutting capabilities. Embarking on this approach can form the basis of a virtuous cycle in which all of the various stakeholders come together.

We believe that, if more widely pursued, this EI approach has potential in efficiently tackling cross-PoR requirements. Furthermore, our findings about the benefits of limiting complexity with short schedules may have real merit for the efforts of more conventional PoRs. When one limits an effort to one year, it automatically changes the assumptions, focuses effort and lowers risks. The relationship between the time allotted to an IT project and the chance that it will not meet expectations has been noted in the commercial world—“the longer a project is scheduled to last, the more likely it is that it will run over time and budget, with every additional year spent on the project increasing cost overruns by 15 percent,” according to a McKinsey and Company report. There also are signs that the U.S. Government already is shifting toward using shorter development cycles as a means for improvement. As Roger Baker, chief information officer of the Veterans Administration, said, “We are huge fans of Agile [development], and are using it in our most critical programs.”


To print a PDF copy of this article, click here.

Kruse, Ph.D., is lead information systems engineer at MITRE Corporation in Bedford, Mass. Slattery is principal multi-discipline systems engineer at MITRE.

The authors can be contacted at wkruse@mitre.org and slattery@mitre.org.

Comments

comments

Leave a Reply

Your email address will not be published. Required fields are marked *