mar-12-article-6-lead

Performance Based Logistics and Project Proof Point A Study of PBL Effectiveness


To print a PDF copy of this article, click here.

Authors: John Boyce and Allan Banghart

There has been much debate recently about performance based logistics (PBL) as a sustainment strategy. Claims about the strengths and weaknesses of PBL have usually been based on emotionally charged anecdotal evidence and opinions, rather than facts.


To address this, the principal deputy assistant secretary of Defense for logistics and materiel readiness chartered a study to perform an independent, fact-based assessment of PBL product support strategies. Called Project Proof Point, the analysis is intended to provide conclusive evidence of the effectiveness and affordability of DoD PBL strategies. A team of subject matter experts from Deloitte Consulting, Supply Chain Visions, and Auburn University, in conjunction with the Office of the Deputy Assistant Secretary of Defense–Materiel Readiness, began the project in September 2010.

Performance based logistics (PBL), also known as performance based life cycle product support, is an outcome-based support strategy that plans and delivers an integrated, affordable performance solution designed to optimize system readiness. Its original intent was to improve weapon system readiness that had been severely degraded. The 2001 DAU publication Product Support for the 21st Century noted, “The emphasis is shifting from the performance of individual stovepipe functions (e.g., procurement; supply; transportation) to harmonizing the functions to improve weapon system readiness.” More recently, attention has been on achieving the optimal balance between warfighter readiness and affordability.

mar-12-article-6-secondaryIt is important to stress that PBL strategies are not synonymous with contractor logistics support (CLS). PBL is about how a system is supported and success is measured. The success of the product support strategy is ultimately determined by its ability to meet the key performance parameter (KPP)threshold value for materiel availability and the key system attributes threshold values for materiel reliability, operations and support costs, and other program-specific supportability requirements. CLS is about who provides the support (whether it is performance-based or not).

The transition from traditional transactional support to performance based support started with the DoD report to Congress on product support reengineering. The first official use of the term was in the 2001 Quadrennial Defense Review (QDR) followed by the DODD 5000.01 and DODI 5000.02, which require that performance based life cycle product support (PBL) strategies be used. PBL was recognized as the best way to ensure every part of the product support package is connected and contributing to the warfighter’s mission capability.

Today there is general agreement that PBL has performed as intended and improved readiness in virtually every application. However, there is a sentiment among some that PBLs are more expensive than transactional alternatives. The Deloitte team’s approach to the analysis was to evaluate the following hypothesis: Sustaining materiel via Performance Based Logistics arrangements delivers improved readiness at reduced life cycle costs. That is, the cost per unit of performance to DoD is lower when a system, sub-system, or component is maintained via a PBL agreement rather than through traditional, transactional maintenance arrangements. The analysis of the sample data supports this hypothesis.

Methodology

The Proof Point team used a two-tiered, fact-based method to test its hypothesis: Sustaining weapon systems, sub-systems and major components via performance based logistics arrangements delivers improved readiness at reduced life cycle costs when compared to traditional, transactional sustainment arrangements.

First tier: A “middle dive” analysis was conducted on 21 weapon systems, sub-systems, and components representing all military Services and varied contract structures to determine what the preponderance of data and facts revealed regarding the impact of PBLs on performance and the cost to sustain equipment. These analyses employed inductive reasoning to draw generalized conclusions from a finite collection of specific observations. Analyzing 21 of the 89 current PBL programs identified by the Services is a sufficient sample size to support generalizations. The premise of the inductive logical approach is that it indicates probability for the conclusion; that is, it suggests truth but does not ensure it. Specifically it will tell you that cost per unit of performance went up or down but does not prove PBLs caused this outcome.

Second tier, Step I: A “financial deep dive” analysis was conducted on six of the twenty-one weapon systems, sub-systems and components, also representing all military Services and varied contract structures, to tighten the proof gap regarding the impact of PBLs on the cost to sustain equipment. Both a financial accounting approach utilizing the OEM’s cost structure and the Service’s price structure, and an in-depth analysis of the negotiation process and OEM’s investment strategies were used to support a suggested linkage between the Performance Based Logistics Strategy and a change in cost.

Second tier, Step II: A “statistical deep dive” analyses was conducted on 5 of the 21 weapon systems, sub-systems and components, also representing all military Services and varied contract structures, to provide definitive point of proof of the impact of PBLs on the cost to sustain specific equipments. Both an inductive approach and a rigorous statistical approach were used. Materiel demand and availability and cost prediction models with generalized linear modeling approaches were used to support investigations of suggested links between the PBL strategy and changes in cost. Using generalized Poisson regression techniques, the team developed a full model of expected demand and availability as a function of materiel, time, and their interaction. From these models, tests for trends and corresponding estimated effects were produced. The overall cost, based on the average cost, was computed as a function of materiel demand and availability. Statistically significant, conservative estimates for the effect of PBLs on cost and associated confidence intervals were computed and are provided.

Results

mar-12-article-6-secondary-2PBL tenet adherence among sustainment arrangements selected for Proof Point analyses spanned the spectrum from strong (but none with 100% adherence) to essentially nonexistent. Of the 21 arrangements reviewed, 18 adhered to strategy tenets in some meaningful ways and are considered PBLs. Three of the arrangements did not embrace PBL tenets in any substantive manner. The weaker results uncovered during the analyses of the three (essentially) non-PBL sustainment arrangements tended to bolster the initiative’s overarching conclusion noted above. Key findings stratified by level of evidence supporting the conclusions:

Statistical results with a defined level of confidence:

  • PBLs can work.
  • PBLs have successfully incentivized PBL provider behavior that delivered superior sustainment pricing and performance for systems, sub-systems and components.
  • Compelling evidence, absent the ability to meet the strictest statistical criteria:
  • PBLs do work (when there is substantive program adherence to PBL tenets).
  • Well crafted PBL arrangements “manufacture competition” by incentivizing companies to compete against internal waste and quality challenges in order to drive up quality (thereby reducing demand) while simultaneously driving down process, labor and material costs.
  • PBL provider behavior is directly linked to the incentives embedded in the arrangement; the military Services set the contractual arrangement.
  • Services get the outcomes for which they contract/incentivize.
  • Preponderance of the evidence:
  • Longer-term contracts that provide assured revenue streams and contain well-crafted cost and performance incentives drive predictably positive outcomes for the Services.

The PBL arrangements that were analyzed clearly reduced DoD’s costs per unit of performance while simultaneously driving up the absolute levels of system, sub-system and component readiness/availability. (See Figures 1 and 2.)

Figure 1. Contract Type

mar-12-article-6-figure-1

Figure 2. Contract Length

mar-12-article-6-figure-2

Due to the proprietary, competition-sensitive nature of the data analyzed, the specifics of the analysis cannot be shared in a public forum. However, an aggregated table of the analysis results is provided in Figure 3. The programs are listed based on an assessment of the programs’ PBL maturity.

Figure 3. Aggregated Analysis Results

mar-12-article-6-figure-3

Of the 21 programs evaluated, 13 began under a non-PBL support strategy, and 12 realized improved operational readiness at a reduced cost, compared with their pre-PBL support. The remaining 8 programs were supported from inception by a PBL strategy and had no pre-PBL data to evaluate. Even so, 17 programs had improved performance and lowered cost over time.

Conclusions

The study concluded that “PBL arrangements which substantially adhere to generally recognized PBL tenets reduce DoD cost per unit of performance while simultaneously driving up the absolute levels of system, sub-system, and major component readiness/availability when compared to non-PBL arrangements.”

It should be stressed that this conclusion holds true independent of individual PBLs’ rigid adherence to all the tenets of an ideal PBL arrangement, exhaustive contract oversight, or contract renegotiation. The consistent ability of PBL arrangements to deliver positive cost and performance results with less-than-strict adherence to all tenets suggests the strategy is robust.

Any business strategy whose success requires flawless execution is destined for failure in the long run.

Although tasked to perform a quantitative analysis, the Deloitte team was able to capture a number of additional qualitative observations as well:

  • PBLs can deliver significant value even with less than perfect implementation.
  • PBLs do not necessarily outsource or degrade DoD’s organic capability. Many PBLs include public/private partnering and have improved organic capability and increased workload.
  • PBLs can work with government providers, but the incentives are more difficult to establish and track.

A few key takeaways for program managers and product support managers alike:

  • PBL product support strategies work. In fact, PBL product support strategies deliver both reduced cost of ownership and increased readiness.
  • PBL strategies are flexible. They are equally effective regardless of whether applied to system, subsystem, or component level product support.
  • PBL strategies are policy. DoD Directive 5000.01, paragraph E1.1.17. directs that “PMs shall develop and implement performance-based logistics strategies that optimize total system availability while minimizing cost and logistics footprint.”
  • PBL strategies are not synonymous with, nor should they be confused with Contractor Logistics Support (CLS). Successful PBL strategies leverage a best value mix of both public and private sector capabilities.

The Department spends more than $90 billion on sustainment every year. A conservative estimate of savings that could result from broadly transitioning to PBL sustainment across the DoD ranges from 10 percent to 20 percent—every year.

‘PMs shall develop and implement performance-based logistics strategies that optimize total system availability while minimizing cost and logistics footprint.’ —DoD Directive 5000.01

Proof Point addressed the cost and performance information gap associated with PBLs. PBL strategy accommodates a wide range of contractual options to address financial flexibility and other concerns. Since military program offices establish and manage the contractual arrangements associated with PBLs, the unanswered question is: “Is the Department willing to forgo 10- to 20-percent savings every year in lieu of deploying a robust, performance based life cycle sustainment program across the DoD?” The answer must be a clear and unequivocal no!

Boyce spent 27 years in the Navy as an aerospace maintenance duty officer, serving in a variety of operational and acquisition positions. He is currently a consultant providing full time support to the deputy assistant secretary of Defense–materiel readiness. Banghart is a senior advisor with Deloitte Consulting and former director of enterprise transformation at the Defense Logistics Agency.

The authors can be contacted at john.boyce.ctr@osd.mil and abanghart@deloitte.com.


To print a PDF copy of this article, click here.

Comments

comments

Leave a Reply

Your email address will not be published. Required fields are marked *