To print a PDF version of this article, click here.
The National Defense Authorization Act (NDAA) for Fiscal Year (FY) 2016 (NDAA) includes many acquisition reforms. The Senate version included a provision that would have required the Secretary of Defense to develop standards, policies and guidelines based on nationally accredited standards for program and project management as well as policies to monitor compliance. The Project Management Body of Knowledge (PMBOK Guide) is the only qualifying standard. The provision was not adopted during conference with the House. However, if the PMBOK Guide were made a contract requirement to replace the Earned Value Management System (EVMS) standard (ANSI-748), any program manager (PM) finally would be able to identify and pinpoint emerging problems on a timely basis and act as early as possible to resolve problems. This article discusses the content and benefits of the PMBOK Guide but also includes guidance for integrating systems engineering (SE) and risk management with EVM independently of the PMBOK Guide.
PM Responsibilities, Needs and Tools
Per Department of Defense Instruction (DoDI) 5000.02, Operation of the Defense Acquisition System, PM responsibilities include achieving the cost, schedule and performance parameters specified in the Milestone Decision Authority-approved Acquisition Program Baseline (APB). Per the Defense Acquisition Guidebook (DAG), the PM should require contractors and government activities to use internal management control systems that “properly relate cost, schedule, and technical accomplishment.” Also, per DAG, “risk management is most effective when fully integrated with the program’s SE and management processes.”
During the Engineering and Manufacturing Development (EMD) phase, the PM must develop, build and test a product to verify that all operational and derived requirements have been met.
The artifacts and tools that a PM needs, per DoDI 5000.02, DAG, and the SE Plan Guide (SEP), include:
- Capability requirements
- Functional and physical characteristics of the system design integrated with the SEP
- Technical baseline
- Product baseline for all configuration items
- System baselines (functional, allocated, product)
- Requirements traceability between the system’s technical requirements and work breakdown structure (WBS)
- Technical performance measures (TPM) and metrics to assess program progress
- Risk management and tracking
- Earned Value Management (EVM)
- Performance Measurement Baseline (PMB)
Unfortunately, there are omissions in the acquisition process, regulations and guides that impede the PM’s success. There are no requirements for contractors to use internal management control systems that properly relate cost, schedule and technical accomplishment with the following tools and artifacts: SEP; Requirements traceability between the system’s technical requirements and WBS; system baselines (functional, allocated, product); incorporation of product baseline into PMB; TPMs; and risk management and tracking.
In 2009, the DoD reported to the House and Senate oversight committees that the “utility of EVM has declined to a level where it does not serve its intended purpose.”
The use of EVM is not sufficient to provide the PM with valid information on cost, schedule and technical performance. Unfortunately, EVM, when implemented by ANSI-748, is not designed to provide performance toward achieving the technical or product baseline.
First, ANSI-748 cites only the “work scope” not the technical baseline or the product scope that is in the APB. Second, ANSI-748 measures only the “quantity of work performed” and not the quality of the system being designed and tested. Third, the use of TPMs in ANSI-748 is optional. Consequently, DoDI 5000.02 is not convincing in its assertions that EVM “promotes an environment … in which problems are identified, pinpointed, and acted upon as early as possible” and also that it is a “powerful program planning and management tool.”
The title of ANSI-748 confirms that it is only designed to manage a statement of work and not a project that concludes with a product. Its title states that it is a “Guide’’ for “Coordination of Work Scope, Schedule, and Cost Objectives.” ”Product” or “technical” objectives are absent.
The purpose of EVM is stated in Office of Management and Budget (OMB) Circular No. A-11, Planning, Budgeting, Acquisition and Management of Capital Assets. Section 300-5 of OMB Circular A-11 states that performance-based acquisition management should be based on the EVMS standard and measure progress toward milestones, cost, capability to meet specified requirements, timeliness and quality.
However, in 2009, the Department of Defense (DoD) reported to the House and Senate oversight committees that the “utility of EVM has declined to a level where it does not serve its intended purpose.” Per the report, the PM should ensure that the EVM process measures the quality and technical maturity of technical work products instead of just the quantity of work performed. The report stated that EVM can be an effective program management tool only if the EVM processes are augmented with a rigorous SE process and SE products are costed and included in EVM tracking.
Now, 6 years later, Congress still is considering EVM as a target for acquisition reform. For more information on the deficiencies of ANSI-748, including the “Quality Gap,” see my article in the November-December 2010 issue of Defense AT&L, “Earned Value Management Acquisition Reform”. (See http://www.pb-ev.com/ndaa-.html or http://www.dau.mil/pubscats/ATL%20Docs/Nov-Dec10/Solomon.pdf.)
A Project Management Standard
EVM, based on ANSI-748, is used primarily by federal contractors when contractually required. A more powerful tool is the ANSI standard that voluntarily is used worldwide because it works, not because it is imposed by federal acquisition regulations. It is the Project Management Institute (PMI) PMBOK Guide.
The needs of the PM that are covered by the PMBOK Guide but absent in ANSI-748 include technical or product baseline; requirements management and traceability; and risk management. The PMBOK Guide contains many artifacts and tools that have no counterpart in ANSI-748, including:
- Product scope description documenting the characteristics of the product that the project will create. It progressively elaborates the product’s characteristics … described in the project charter and requirements documentation.
- Project scope involving the work that needs to be accomplished to deliver a product … with the specified features and functions.
- Requirements documentation provides the requirements baseline; it is unambiguous (measurable and testable), traceable, complete, consistent, and acceptable to key stakeholders. Components include functional requirements, nonfunctional requirements, quality requirements and acceptance criteria.
- Requirements Management Plan includes product metrics that will be used.
- WBS Dictionary includes quality requirements, acceptance criteria.
- Scope Baseline includes product scope description, project deliverables and defines product user acceptance criteria.
- Control Scope or the process of monitoring the status of the project and product scope and managing changes to the scope baseline. Completion of the product scope is measured against the product requirements.
- Requirements Traceability Matrix (RTM) includes requirements to project (including product) scope/WBS objectives, product design, test strategy and test scenarios.
- Conduct risk management planning, identification, qualitative risk analysis, quantitative risk analysis, response planning and controlling risk.
The PMBOK Guide also covers EVM topics such as scheduling (including network diagrams), PMB, control accounts, work packages, earned value, variance analysis, estimate at completion, and management reserve.
PMBOK Guide Deficiencies
Some ANSI-748 guidelines have no equivalent in the PMBOK Guide. These relate to organization costs, material accounting and unit/equivalent/lot costs. It is recommended that, during the acquisition reform reviews of existing regulations, these guidelines be considered for elimination.
Use of TPMs also is optional in the PMBOK Guide. Consequently, any revision to the acquisition policies and regulations should require contractors to identify and use TPMs.
Acquisition reforms should include requirements for the PM and contractors to use PMBOK Guide for EMD contracts that are above specified threshold values. The PM finally will have valid information and tools needed to properly relate cost, schedule and technical accomplishment; manage risk and achieve the contract’s cost, schedule and performance parameters.
A PM can ensure integration of technical performance with EVM even if the PMBOK Guide is not utilized. However, there must be a contractually required SEP with linkage of SE work products—such as the requirements in the RTM and TPMs—with the Integrated Master Schedule and work packages.
Effective acquisition reform is a stated objective of DoD and of the chairmen of the Senate and House Armed Services Committees. The PMBOK Guide and SEP currently are “guidance.” It is recommended that the actual reforms impose the “guidance” provided above as contractual requirements.
Solomon is retired from Northrop.Grumman Corp. He is a Project Management Professional and co-author of ANSI-748 and the book “Performance-Based Earned Value,” published in 2007. Solomon is a 1998 recipient of the Defense Department’s David Packard Excellence in Acquisition Award. The author maintains a project management Website, www.pb-ev.com, and welcomes comments and questions.
The author can be contacted at paul.solomon@pb-ev. com.