What’s in Your Estimate at Completion?


To print a PDF copy of this article, click here.

Authors: Pat Barker and Roberta Tomasini

In this era of increased emphasis on cost constraints and affordability, we often ask, “How much will this cost?” Within an Earned Value Management (EVM) context, the answer is found in the Estimate at Completion, or EAC. EACs are discussed in various venues from a technical manager’s office at a prime contractor facility to a crowded conference room at the Pentagon. All eyes become fixated on “the number.”

It is important for government program managers (PMs) to realize, especially as budgets tighten, that the EAC is more than just a number. The EAC is a gateway to insight on past, present and future program performance. The EAC numbers are the tip of an iceberg. Below the surface is a rich story describing why the numbers are what they are, how they were derived, and what they reflect. EACs provide insight and contributions to both government and contractor planning and management process execution.

Used as a leading indicator, an EAC provides a PM an opportunity to make proactive decisions. It is a pathway to explore not only whether a provider is likely to deliver on its promise, but whether the government PM can deliver within the cost, schedule, performance (C/S/P) constraints of the program’s Acquisition Program Baseline (APB). EAC discussions in the government program management office (PMO) are also excellent chances for the government PM to listen to what his or her staff thinks.

EAC Insights

EACs are tied directly to the contract portion of the program threshold and objective costs in an APB. In addition, EACs provide a current estimate against the contract portion of the schedule and performance parameters in the APB. EACs therefore become a key consideration in exploring APB trade space with respect to C/S/P, to retain program affordability. Thus, EACs tell a story of real program integration.

article-8-secondary-1EACs help clarify the influence of contractor overhead costs, general and administrative costs, and cost of money. Add in profit/fee and you get price, which takes you directly into budget territory: totals, phasing, obligations and expenditures. Thus an EAC discussion should be married with evaluation of a contractor’s Contract Funds Status Report (CFSR) to assess whether the current time-phased budget and execution plan will support the latest EAC.
Given that usually two-thirds of a program’s total ownership costs (TOC) reside ”downstream” in the program life-cycle process, ”upstream” EACs become important leading indicators of ultimate costs. Significant EAC changes demand near-immediate revisits to TOC estimates.

EACs also should reflect the reality of performance trends and forecasting. It is of paramount importance to compare EAC results with indices such as current and projected cost variance, schedule variance, Cost Performance Index (CPI), Schedule Performance Index (SPI) and Variance at Completion (VAC). Every EAC drives a particular To-Complete Performance Index (TCPI), which is an important additional ”reality check.”

The derivation and reporting of a contractor EAC typically is associated with a detailed “bottoms up” analysis of work to go, which takes significant planning and assessment effort on the part of contractor control account managers (CAMs). Some organizations update their “bottoms up” EACs annually, others semiannually, some even more frequently. EACs also are expected to be assessed frequently (monthly) by each CAM via less-detailed means (in terms of planning) but with thorough consideration of performance to date using a variety of metrics and indicators.

The credibility of EACs can also be reflections of incentives, culture and/or trust. There have been cases of EACs “adjusted” based in part on a desired story or outcome rather than entirely on performance to date, work to go and associated risk. For further exploration, see Table 1.

Table 1. Warning Signs for EVM (EAC) Reporting Issues

something you might see that should “raise a flag” What it might mean (and is worth a question or two for clarification
A PM deferring to the “cost lead” or “EVM lead” to explain an EFC Lack of PM comfort with EAC derivation, lack of PM familiarity with EVM
EACs stagnant, not being updated at regular intervals EAC development/assessment process that is either immature or atrophied. Lack of PM attention to EAC.
A volatile and “ipwardly accelerating” EAC (changes significantly each month) Weak (and disconnected) program planning, scheduling and EVM activities. Lack of anticipation and proactive decision-making.
Best Case (BC), Worst Case (WC) and Most Likely (ML) EAC are equal Significant EVM oricess execution prblem. Complete lack of risk/opportunity integration. Lack of understanding of what EAC represents.
ML EAC < BC EAC EAC development/assessment process that is either immature or atrophied. Lack of PM attention to EAC.
ML EAC > WC EAC EAC development/assessment process that is either immature or atrophy. Lack of PM attention to EAC.
Each of near-exact agreement between contractor EAC and government EAC Government staff lacks comfort, experience with EVM. Likely lack of integration/interface between government EVM analysts and rest of government staff, especially technical staff.
Government EAC(s) consist solely of “Gold Card” formula results with little or no accompanying explanation/rationale. Government staff lacks familiarity with how to integrate risk and EVM. Likely lack of integration/interface between government EVM analysts and rest of PMO staff, especially technical staff
Lack of robust explanation of EAC cost drivers Lack of comfort, experience with EVM, and especially risk and EVM. Likely disconnect between integrated Master Schedule, Schedule Risk Analysis (SRA) and EVM.
Exact or near-exact match of EAC to prominent cost-related contract elements such as BAC, TAB, Negotiated Price, PMO budget EAC development/assessment process that is either immature or atrophied. PM lack of comfort wiht EAC derivation and/or lack of familiarity with EVM. Could reflect organizational culture of risk aversion. Could also reflect budget constraints.
Sustained high SPI Premature claim of work accomplished. High percentage of “level of effort? earned value technique.
Sustained high CPI High percentage of “level of effort” earned value technique combined with understaffing and/or substitution of lower labor rate personnel for higher rate personnel. Premature claim of work accomplished. Premature opening of work packages.
Contractor expenditures different than anticipated Contract Funds Status Report (CFSR) misalignment with reported Actual Cost of Work Performed from CPR/IPMR. Must reconcile CFSR (with fee) to CPR/IPMR (without fee).
No reference in RAC to schedule forecast derived from SRA Disconnect between scheduling discipline. EVM ad risk management. Lack of cost adjustment to pay for “standing/marching army.”
To-Complete Performance Index (TCPI) associated with the EAC is greater than the cumulative Cost Performance Index (CPI) reported to date EAC might be unrealistic. A TCPI exceeding the CPI by 5% warrants pointed questions, and a 10% or greater difference warrants concern. It is rare for performance efficiency to increase by such a large margin unless fundamental changes to program management and execution are assumed.

EACs Should Capture Risk

Most importantly, EACs give us a healthy look at risk assumptions.

There have been cases of EACs “adjusted” based in part on a desired story or outcome rather than entirely on performance to date, work to go and associated risk.

 

Contractor-derived EACs are part of formal reporting artifacts such as the Integrated Program Management Report, or IPMR (formerly called the Contract Performance Report or CPR). Government PMO-derived EACs are intended to be compared to the contractor EACs. There ought to be some reasonable degree of traceability in each, from the initial assumptions through the derivation and into the final result.

Of all potential factors, the most significant differences between contractor and government EACs lie in risk and opportunity assumptions, which mean there is a range of potential EACs, typically reported as Most Likely (ML), Best Case (BC) and Worst Case (WC). The ML is the most commonly reported EAC, and is exactly what it sounds like: the ”best guess” final result after considering all dynamics, risks and opportunities. A WC is the result anticipated if most risks become issues and few opportunities, if any, are captured. By contrast, a BC reflects most risks not materializing and most opportunities being capitalized upon (i.e., things mostly going right the first time). The range, or spread, among these EACs reflects directly on the uncertainty associated with the program, starting with the first EAC on Day One of program execution. From a cost estimator’s perspective, the range between BC and WC is typically the broadest toward the beginning of a program and typically the narrowest toward the end. The PM therefore should consider the cost estimator’s perspective relative to EACs and their program budget. In addition, the PM can use the EAC range to be continually apprised of risk and opportunity and how much of each is assumed in the most likely EAC.

A “90-Second Back-of-the-Envelope” PM EAC Cross-Check

Many EVM training seminars and courses within and outside the Department of Defense (DoD) encourage students to focus on standard formulas when it comes to calculating EACs. The Defense Acquisition University EVM “Gold Card” is one of the most well-known references for widely accepted EAC formulas. One chooses a formula based on local conditions of cost performance, schedule performance and/or degree of program completion. As a general rule, these formulas have two components—actual costs to date and anticipated costs going forward. These anticipated costs are derived through dividing the budget of work remaining by an efficiency factor reflecting schedule and/or cost performance to date. While these formulas are effective and highly capable of forecasting, they are not necessarily geared to foster a direct conversation on risk.

Therefore, this article offers a “90-second back-of-the-envelope” formula for government PMs. This formula is a simple cross-check that enables the government PM to discuss key entering arguments of a proposed EAC. This “back-of-the-envelope” formula and explanation of its terms is below. A specific example is shown in Table 2.

Table 2. Back-of-the-Envelope Example

Row Term What a contractor might report The Government PM’s “Back-of-the-Envelope” Notes Notes on government PM’s Actions
1 Actual Costs to Date $50 million $50 million Extracted directly from Format 1 CPR/IPMR
2 Budget of work to go (BAC-BCWP) $50 million $50 million Extracted directly from Format 1 CPR/IPMR
3 MR $3 million Extracted directly from Format 1 CPR/IPMR
4 Risk/Opportunity $5 million (factored) $8 million (dollarized risks only) Extracted directly from contractor Risk Register and/or Format 5 CPR/IPMR
5 Marching Army $6 million Extracted from contractor SRA results for contract end date and current “burn rate” from CPR/IPMR. This is based on a 6-month contract end date slip at a burn rate of $1 million per month
6 Total $105 million $117 million  
A Program Manager’s 90-Second
“Back-of-the-Envelope” EAC
Costs So Far Budget to Go Remaining Management Reserve Dollarized Risks Marching Army Costs

90-Second Formula Components Defined

Each of the 90-second formula components is defined as follows:

Costs So Far: An EAC should include program costs to date. These otherwise are known as “actual costs,” “actuals” or ACWP (Actual Costs of Work Performed). They represent “sunk costs.” The PM starts here by recording ACWP as reported in the CPR/IPMR.

Budget to Go: Second, an EAC should consider the budget for the remaining work. This is calculated by taking Budget at Completion (BAC) and the cumulative value for Budgeted Cost of Work Performed (BCWP) from the latest CPR/IPMR (i.e., BAC minus BCWP cumulative). This value is added to the ACWP.

Remaining Management Reserve (MR): DoD acquisition history for medium- to high-risk programs tells us odds are a contractor has planned insufficient MR and will consume whatever it does have. Thus, we add all the remaining MR. The MR value can also be found in the CPR/IPMR.

Dollarized Risks: When it comes to quantified cost risk, contractors put a dollar value against each risk. These are dollarized risks. Furthermore, some contractors assess a probability value against each risk. They then develop a factored risk by multiplying the dollarized risk value by the probability. The resulting number is added into the EAC. By contrast, the back-of-the-envelope approach adds the full dollarized (not factored) risk value. The assumption here is that, though many risks may not be realized, other risks will continue to emerge until the end of the contract. Total dollarized risks can be found on the risk list or risk register. They also may be found in Format 5 of the CPR/IPMR. If there is not a dollarized risk list, try to get a rough estimate of the total value.

Marching Army Costs: When the contract end date schedule slips, everything slips, but people typically still get paid and overhead dollars still accrue. Multiplying anticipated contract end date schedule slip (in months) by an estimate of the contractor’s “burn rate” when program slip is anticipated will do the trick for this term. A quick way to select the burn-rate value is to use the ACWP of the current period from the CPR/IPMR.

Discussing the 90-Second Formula Components

These back-of-the-envelope formula components are not the final word for an EAC. Rather, each sets the stage for a robust and systematic EAC discussion.

Costs So Far: The reported actual costs might not truly reflect sunk costs. In some cases, portions of the reported actual costs are, in fact, estimated costs. This presents an opportunity for the PM to clarify the impact, if any, of these “estimated” actuals. For example, prime contractors typically utilize a large array of subcontractors, and subcontractors employ subcontractors. Accounting periods and invoicing across these “layers” often are not synchronized. The final “actuals” number likely could differ from the estimated number.

Budget to Go: Work remaining directly reflects work accomplished to date. A PM uses this term to discuss the contractor’s measurement of “percent complete.” How reasonable and/or robust is that measurement? What is really meant by 35 percent complete, 40 percent complete? An integrated master plan (IMP) becomes an excellent supporting tool for this part of the discussion.

Remaining MR: This term kicks off the program risk discussion. How did the contractor derive the MR? Was it pulled “across the board” using some factor like 8 percent or 10 percent (i.e., a management challenge) of negotiated cost? Does the factor differ between material and labor? How does this MR reflect the risk register content and nature of the work? How has the contractor been burning MR to date and for what purpose? How does the contractor factor into risk handling and forecasting? Has ”disbursed” or ”allocated” MR been reflected directly as work in the integrated master schedule (IMS)?

Dollarized Risks: This is an excellent opportunity to ask the hard questions about risk (and opportunity) assumptions and impacts. In particular, it sheds light on how risks actually are quantified in different organizations. The end result should be a mutual understanding of risk perspectives, even if the parties differ in their bottom line.

Marching Army Costs: Time is money, so this is an excellent opportunity for the PM to inquire how the contractor schedule risk analysis (SRA) was conducted, and to what degree its results influenced the EAC. An SRA accomplished in a manner consistent with industry and DoD expectations is an exceptionally powerful tool for PM insight and proactive decision-making.

Factored risks often are derived via the ubiquitous “risk cube.” However, that particular risk management tool is qualitative, not quantitative.

Points of Order

A concern with this approach might be, “Wait a minute: MR plus dollarized risk plus schedule burn (marching army)? You are double and even triple-counting risk!” Where highly disciplined approaches to program management are encouraged and practiced, a “double” or “triple” count concern might be warranted. This formula assumes that this exceptional degree of discipline is not the case in most organizations.

article-8-secondary-2It is often observed that not all risk considerations are created equal. MR, for example, often appears as a top-down “challenge” to managers, an across-the-board percentage cut at the beginning of the program. Derivation of MR levels and risk management assessments often are performed in separate venues and times. As a program’s risk management process spins up, risk impacts and mitigation costs are developed in various ways and often are loosely connected to MR levels.

In practice, the ubiquitous “risk cube” tends to be used as a basis for quantitative risk calculations. However, that particular risk management tool is qualitative, not quantitative. One person’s “high probability” is another person’s “low probability,” and it is hard to distinguish between how 35 percent versus 45 percent versus 55 percent was derived. The same goes for assessment of impact. It is nearly impossible to predict program-level risk impacts without the aid of program-level models such as a program cost model and an IMS. In short, true quantitative risk analysis results, factored into no-kidding quantitative numbers such as EACs, need more than a risk cube. A back-of-the-envelope tool merely opens the door for that discussion.

Best practice to the contrary, risk decision points, handling approaches and forecasting often are not directly reflected in the IMS in a way that models their respective influence on the program. Not all programs perform SRAs, and those that do so will not always integrate the results into the risk register, let alone the EAC. This 90-second approach
delineates schedule risk and other risk considerations separately not because that is how it ought to be, but instead as a reminder to PMs to bring IMS considerations to the forefront of EAC discussions.

Opportunities could, and should be, assessed in a detailed EAC review. However, we assume strictly for our “back of the envelope” purposes that people are optimistic by nature and therefore we focus on risk and do not factor in opportunities, per se. Therefore, this approach fosters a discussion on opportunity (within which resides the topic of ”should cost”).

The “Blind Side”

Acute awareness of the EAC and its derivation is important beyond the government and contractor PM conversation, because they aren’t the only ones who see, and interpret, an EAC. While we know EACs typically are reported within a contract through a deliverable such as an IPMR, EACs also find their way elsewhere. The contractor PM has his or her own command chain, all the way up to the chief executive officer, that will have an interest in—and influence upon—the EAC. On the government side, the EAC finds its way into program dashboards and briefings “up the chain” to program executive offices and acquisition executives in the Service or agency and in the Office of the Secretary of Defense (OSD). EACs are contained within Defense Acquisition Executive Summary (DAES) quarterly reports and DAES briefing charts. Selected Acquisition Reports contain EACs. EAC information gets pulled into other databases such as the Central article-8-secondary-4Repository at the OSD level. The Defense Contract Management Agency (DCMA) assesses and reports on EACs. In short, EAC discussions take place in venues beyond the government PM’s zone of control, and lack of awareness can result in a PM being ”blindsided” and unprepared at a most inopportune time.

The PM is expected to know the EAC and what it represents. By asking targeted and insightful questions about contractor EACs, he or she can help make EAC generation and review more efficient and effective. A “back of the envelope” approach enables tremendous insight into how the contractor accommodates risk within its performance measurement and forecasting functions and is one way for a PM to assess whether risk has been reasonably factored into a contractor’s EAC. Smart questions can “cut to the chase” quickly and discover elements or indicators that reflect process issues, clarify perspectives, prevent late-to-need information and foster proactive decision-making. A 90-second EAC is not the last word on a program EAC. It is but a cross-reference for a PM, a starting point for discussion, and a catalyst for more detialed government PMO staff analysis. Powerful EAC questions and robust EAC discussions put “affordability” in the forefront in a very real and productive way.


To print a PDF copy of this article, click here.

Barker and Tomasini are professors at Defense Systems Management College (DSMC).

The authors can be reached at patrick.barker@dau.mil and roberta.tomasini@dau.mil.

Comments

comments

Leave a Reply

Your email address will not be published. Required fields are marked *