To print a PDF version of this article, click here.
Conceptual models based on economic and operations research principles can yield valuable insight into defense acquisition decisions. This article focuses on models that place varying degrees of emphasis on each objective of the defense acquisition system: cost (low cost), schedule (short cycle times), and performance (high system performance). The most appealing conceptual model is chosen, which the authors posit that, if adopted, would lead to shifts in priorities that could facilitate better outcomes, as empirical results suggest. Finally, several policy prescriptions implied by the model are briefly explored.
Principles from microeconomic theory and operations research can provide insight into acquisition decisions to produce military capabilities in an environment of scarce resources. To begin a discussion of the analytic models involved, it helps to identify desired outcomes. In the acquisition literature, cost (low cost), schedule (short cycle times), and performance (high system performance) are generally regarded as primary objectives in fielding new systems (Department of Defense [DoD], 2006). This article focuses on conceptual models involving these three goals that place varying degrees of emphasis on each objective. The most appealing conceptual model is chosen, which we posit that, if adopted, could clearly lead to badly needed shifts in priorities. Empirical results bolster the proposition that changing priorities could lead to better outcomes. Finally, we discuss several policy changes implied by the chosen model.
In pursuing a defense acquisition, the immediate question becomes what combination of cost, schedule, and performance can or should be considered optimal? If the only guidance provided to analysts is to do their best to minimize cost and time to field while maximizing performance, then making tradeoffs will rely on professional military judgment at best or become arbitrary at worst. The basis for decision making can be unclear and result in disputes, and the acquisition professional may lose sight of the overall objective. Therefore, the goal of this article is to develop a new way of characterizing the acquisition problem that will help decision makers make more informed tradeoffs.
The Efficient Frontier of Defense Acquisition
Each defense acquisition program can be judged by how much input (time and money) is consumed to produce the desired military output (performance or capability). This uncontroversial statement of defense acquisition system objectives reveals how this incredibly complex system reduces to a relatively simple problem involving production economics. To put it in mathematical terms, a system’s performance can be expressed hypothetically as a function of the independent variables time and money, as described in the following associated interactions.
Performance = f (time,money)
Figure 1. Notional Performance Isoquants
Figure 2. The Defense Acquisition Efficient Frontier
Using this function, performance can be plotted as a function of either time or money in a standard Cartesian graph. Another way to visualize this function is to hold performance constant while varying time and money. All of the combinations of time and money result in the same performance makeup of what is called a performance isoquant. Figure 1 illustrates this concept with two notional performance isoquants. If cost and time are then divided by performance and graphed in the same manner, it becomes clear that increases in either cost/performance or time/performance are undesirable. The efficient frontier of cost, schedule, and performance represents optimal outcomes under the current system. The curve in Figure 2 illustrates the efficient frontier, and points above the line are considered inefficient.
The goal of defense acquisition system managers and practitioners would be to choose to develop and produce systems that occupy the efficient frontier. Although reaching the efficient frontier may, in practice, be extremely challenging, improvements toward that end are constantly sought. As such, a commonly occurring issue arises when one considers how a hypothetically inefficient program should be improved to reach efficiency—should costs be cut, time-to-launch condensed, and/or performance enhanced? In other words, how should marginal resources be allocated?
If the system is allowed to function with limited oversight, how might the actors behave and what are the implications of their decisions? What might be a better way to state the acquisition problem to develop a commonsense approach to the question of how to make tradeoffs between cost, schedule, and performance? Finally, do empirical results tend to support or refute the recommended model, and what sort of actions can be taken to make the current defense acquisition system more consistent with the chosen conceptual model?
Economic Incentives of Defense Contractors and the Defense Department
Edmund Conrow (1995) developed an excellent microeconomic framework to investigate the incentives of buyers and sellers in the defense acquisition system. His most important insight is that while government and contractors have objectives that can and often do conflict, negative outcomes associated with aligned government and contractor incentives are quite likely to occur—and have occurred in the past. To avoid similar outcomes in the future, mitigating incentives or controls must be put in place.
The government and contractors’ preferences can be discussed in terms of cost, schedule, and performance. Clearly, the government prefers low cost, shorter cycle times, and high performance. Meanwhile, contractors’ preferences are more nuanced. In the absence of a fixed-price contract, contractors generally prefer rising system costs to strengthen their own bottom lines. However, this preference is not absolute because spiraling costs can result in negative reviews on the Contractor Performance Assessment Reporting system. Developing higher performing systems improves a contractor’s future market position by keeping it on the cutting edge of technology. Finally, extended schedules imply a longer relationship with a government client that may offer future system development opportunities, though this inclination can backfire when taken too far, as with preferring high system costs. As depicted in Conrow (1995), this can be illustrated graphically using government and contractor indifference curves sketched in cost-performance space.
Figure 3 provides a visual representation of the indifference curves inspired by Conrow (1995). The government’s utility curves are labeled UGi where increasing i indicates increasing utility. For the government, utility clearly increases from right to left and bottom to top. Meanwhile, contractor utility indifference curves are labeled analogously by UCi. The technical possibilities curve represents what performance levels are possible at each cost and intersects both sets of indifference curves at two places, A and B. These are feasible and efficient choices for cost and performance. Meanwhile, point C is infeasible due to lack of adequate technology, while point D is inefficient and, thus, inferior to points A and B. Of the two feasible and efficient points, point B dominates because it offers both government and contractor a higher level of utility.
Figure 3. Government and Contractor Cost and Performance ISO-Schedule Curves with Technical Possibility Curve
This economic analysis reveals that, in the absence of strong cost control, the program office and contractor will tacitly conspire to increase performance at the expense of cost. That is, the program office and the supplier can agree on performance, but they cannot agree on either cost or schedule. Therefore, the natural inclination is to maximize performance and see how much cost and schedule can be massaged. This insight from economic theory reinforces the beliefs of defense experts. Indeed, since program managers and executive officers clearly benefit from managing larger and more complex programs, some claim that the budget and schedule objectives are not as important in the eyes of important decision makers. Laws like Nunn-McCurdy are intended to mitigate this tendency to let costs and schedule spiral out of control. However, some argue that Nunn-McCurdy has failed in its stated goal because it only requires congressional notification of a breach (Ewing, 2011). Often, Congress is already aware of the program’s struggles and does not react to the breach. Further steps need to be taken to effectively discipline the system. A new way to look at the objectives of developing and fielding a system might help clarify what needs to be done, and is the focus of this article’s final section.
A Constrained Optimization Approach
Reconciling the competing objectives of the defense acquisition system has proven to be quite challenging. While optimization and tradespace tools are useful, senior leaders bear primary responsibility for communicating DoD acquisition strategy through their statements and actions. A notional optimization problem specification can be helpful in framing a strategy in a manner that is internally consistent, and such that the goals (referred to as the objective function in operations research) and constraints reinforce one another.
In the language of operations research, any proposed system must not exceed a maximum budgeted cost; take too long to field and, thus, sacrifice military utility; or fall below minimum performance standards. In mathematical terms:
Cost ≤ C
Time ≤ t
Performance ≥ P
As in the previous section, these constraints, taken in isolation, do not offer any rationale for making tradeoffs. For instance, what should be done if the system’s development schedule slides to the right—should costs be increased by allowing more overtime or hiring more staff, or should performance goals be sacrificed to tighten the schedule? Adoption of a decision rule can help facilitate such decisions. Even the adoption of a heuristic decision rule can enlighten analysts and prove useful in promoting the goals of the acquisition system. That is, even though this conceptual model is admittedly a greatly simplified representation of an extraordinarily complex system, it still yields valuable insights to those in defense acquisition leadership positions. Indeed, the models discussed throughout this article assume decision makers possess more information than is ever available in practice, so much is left to professional judgment, particularly regarding unforeseen cost and schedule risks.
This set of constraints suggests a constrained optimization problem specification could be useful in framing the discussion. Standard operations research principles dictate that three different system specifications are possible. The potential explanatory power of each will be judged by the implications arising when the specification is translated into English.
Budget Specification Model
subject to Performance ≥ P
Time ≤ t
The Budget Specification Model can be considered as an attempt at obtaining defense “on the cheap.” Its appeal is intuitive, particularly in today’s budgetary environment of increasing fiscal constraints, but this specification can be removed from consideration because the cost uncertainty associated with it would severely disrupt the planning, programming, budgeting, and execution process. Budgets must be specified precisely, and including the program budget as a constraint is the only feasible way to proceed.
Capabilities-Based Specification Model
subject to Cost ≤ C
Time ≤ t
The Capabilities-Based Specification Model formalizes the undisciplined incentive structure outlined in the previous section. Of the three approaches, this model is the most vulnerable to requirements creep. While this specification will almost certainly lead to the highest performing systems, it will leave the DoD vulnerable to former Secretary of Defense Robert Gates’ critique of “running up the score” (Thompson & Karon, 2009). Furthermore, the two constraints often seem to be somewhat negotiable. Timelines are lengthened to lower short-term costs and expenditure becomes important, seemingly, only when Nunn-McCurdy limits are in danger of being breached. As previously mentioned, even a Nunn-McCurdy breach often fails to motivate change. In other words, the constraints are not binding, leading to an undisciplined system. If this specification is chosen, the focus must be on disciplining the system through enforcement of the constraints and ending requirements creep, for example. With primary emphasis on maximizing capability, there will always be a strong tendency to extend timelines to get that last bit of performance.
Threat Specification Model
subject to Performance ≥ P
Cost ≤ C
The Threat Specification Model best reflects the thinking behind amendments to the Federal Acquisition Regulation for needs of “unusual and compelling urgency” (General Services Administration, 2005). The naming of the threat specification arises from the idealized case: when a threat is imminent and a near-term response is required. This sort of rapid acquisition authority is designed to meet immediate warfighter needs. Some experts have advocated a similar system be adopted for all acquisition. In the words of retired Air Force Lieutenant General David Deptula, “…we need to be able to operate much quicker and inside our adversary’s decision rate” (Hoffman, 2010).
This problem specification has intuitively appealing implications–and it is not susceptible to Secretary Gates’ “runnin’-up-the-score” criticism. Using time as the objective function will, in most cases, reinforce the cost constraint somewhat because simplifying the acquisition process will undoubtedly reduce costs. In any case, in nonemergency acquisition, this objective function and its associated constraints are in harmony to a greater extent than the capabilities-based specification. The primacy of performance in the capabilities-based specification naturally leads to higher costs and longer cycle times, while enhanced focus on reducing cycle times should not cause overall costs to rise to the same degree as long as the concept is not taken too far. The DoD’s Better Buying Power 2.0 initiative (Kendall, 2012) makes the mutually reinforcing nature of reducing both cycle times and cost explicit during the development phase:
This initiative will assess the root causes of long product cycle time, particularly long development cycles, with the goal of significantly reducing the amount of time, and therefore cost, it takes to bring a product from concept to fielding. (pp. 5-6)
Furthermore, the capability constraint is unlikely to be violated, or the system is likely to be cancelled due to inadequate military utility. Because of the objective function’s primacy, requirements creep should not be as ubiquitous. Therefore, this approach should have the added benefit of being easiest to discipline.
An immediate critique of the Threat Specification Model is the possibility of being stuck with less capable systems for many years. If the acquisition cycle is shortened, this criticism is blunted somewhat because our armed forces do not have to be impaired as long with less capability, but it still remains troubling. The most plausible solution is a hybrid framework that follows this decision rule, but retains some of the virtues of the existing system. While the chosen model does not advocate for a single-minded focus on the schedule, program managers should avoid roadblocks that might stymie a program in the absence of overwhelming cost or performance benefits.
While the recommended theoretical framework may be intuitively appealing, the more important question is whether any empirical evidence exists to support the hypothesis that an enhanced focus on schedule will improve acquisition outcomes more generally. Since the available historical data are merely observational, obviously there will be no perfect test. However, the results of a few statistical tests offer some hope of improving acquisition outcomes through adopting a different mindset.
A potentially contentious argument made in advancing the merits of the Threat Specification Model described in the previous section, was that an enhanced focus on the schedule would also pay dividends by helping to avoid unpleasant cost outcomes. This proposition can be examined by determining whether schedule problems precede or occur simultaneously with cost overruns. If schedule slippages occur first, this suggests that, in some cases, cost overruns could be prevented by first ensuring a program can meet its schedule. Of the 54 programs listed as incurring Nunn-McCurdy cost breaches on the Defense Acquisition Management Information Retrieval system from December 1997 to June 2012, 38 (or 70.4 percent) experienced Acquisition Program Baseline schedule breaches in a previous Selected Acquisition Report. If the timing of the breaches is determined by chance, schedule breaches should take place before cost breaches about 50 percent of the time. If we consider these programs to be representative of all programs, this proposition can easily be tested statistically using a simple exact binomial test. Carrying out such a test reveals that the observed results would be very unlikely to occur due to chance (p-value < 0.001), so the unavoidable conclusion that schedule slippages more often precede cost breaches than the converse is true. If the same proposition is tested while allowing for simultaneous breaches, the results are even more conclusive—43 of 54 schedule breaches occurred before or simultaneously with cost breaches (p-value < 1.0×10-5). These test results are presented in Table 1. Such outcomes strongly suggest that schedule problems precede cost breaches. Therefore, preventing the schedule breach could eliminate the cost breach too.
Table 1. Timing of Acquisition Program Baseline Schedule Breaches vs. Nunn-McCurdy Cost Breaches
|Schedule Breach First||70.4%*|
|Schedule Breach First/Same Time||79.6%*|
The next question that arises is whether focusing on schedule can also benefit performance outcomes. This is much more difficult to test since performance goals must clearly be met within a certain timeframe—making the two goals virtually indistinguishable. The key here is to reverse the direction of causation and set performance goals that are realistically within reach with existing technology or imminent scientific advancement. That is, it is important to avoid reaching for unrealistic technological advances to achieve higher performance goals. This proposition has been supported empirically through analysis of Selected Acquisition Reports and Defense Acquisition Executive Summaries (Dacus, 2012). Before DoD acquisition Milestone B, each critical technology element of a program is assessed to determine how far along the discovery process each element has progressed. Low technology readiness levels, which describe the maturity of these critical technology elements, have been linked to larger cost overruns and more pronounced schedule slippage. If two simple decision rules that require minimum individual and system-wide technology readiness levels before a development effort is allowed to become a program of record are enforced, cost and schedule outcomes are improved dramatically, with mean differences in outcomes that are statistically significant (Table 2). These results provide strong evidence that setting more realistic performance goals can lead to substantial improvements in cost and schedule performance. Therefore, performance goals should be set with the schedule very much in mind.
Table 2. Application of the Minimum Technology Readiness Levels Decision Rule
|Quantity of Interest||No Violation||Violation|
|Mean Months Behind Schedule**||7.7 mos.||31.2 mos.|
|Mean Percentage Cost Overrun*||3.2%||35.5%|
The Threat Conceptual Model (also referred to as the Threat Specification Model) lends itself to interpretation through various policy recommendations. Some of these recommendations and a discussion of their implications follow. These examples capture the spirit of the Threat Conceptual Model, but do not represent an attempt to catalog all of its implications. Creative implementation of the values inherent in the model may lead to unanticipated benefits. With that caveat in mind, schedules could be shortened by focusing on stability, by avoiding technology overreach, by keeping systems as simple as possible, and by initiating fewer programs.
Insist on Stability
Continual change, whether in the requirements themselves or in available program funding, leads to extended schedules and should be minimized to emphasize meeting the program schedule. Requirements instability can stem from a desire to improve performance above what was originally envisioned, but consistently engaging in requirements creep conflicts with the proposed mindset. According to those surveyed for the 2006 Defense Acquisition Performance Assessment Report, requirements instability was the most commonly identified reason for cost growth and schedule extensions (Kadish, 2006). According to a 2008 Government Accountability Office report, Defense Acquisitions: Assessments of Selected Weapon Programs, cost increases for programs with no requirements changes after system development were significantly lower (72 percent vice 11 percent). Further, according to a 2011 GAO study, requirements changes added 5 months to delays already being experienced (GAO, 2011). Intuitively, this makes sense; changing direction after the program is underway is likely to increase the time to field and raise costs.
According to those surveyed for the 2006 Defense Acquisition Performance Assessment Report, requirements instability was the most commonly identified reason for cost growth and schedule extensions (Kadish, 2006).
Just as adding requirements tends to cause a later delivery of promised capability, so does a slip in a program’s budget. This often stems from a desire to reduce costs in the short run or from shifting priorities. Of course, some budget changes may be unavoidable due to congressional action, but the Services must make a commitment to the schedule and exercise discipline when possible to avoid disrupting the program. If requirements are to be achievable, there must be the financial resources to pay for them. As it stands, every year programs are forced to rejustify themselves to virtually every funding authority within the program’s funding hierarchy. During that process, Congress may decide to adjust the procurement quantity, as it did with the C-17 when additional numbers were added to the planned buy, creating some second- and third-order effects to other programs. More often, Congress decides to cut the number of program units, as was the case with the F-22. Unfortunately, reduced quantity increases the per-unit cost because the sunk development costs are allocated across fewer units (Deloitte, 2009).
Shifting requirements and budget instability often work together to lengthen programs. Deloitte (2009) noted that:
…most programs are funded and launched while there is still significant uncertainty about everything from systems and technologies to integration, interoperability, and supply chain requirements. This lack of certainty and knowledge makes it difficult or impossible to make informed funding decisions, which often leads to cost overruns and schedule delays. (p. 12)
The DoD begins programs with too many unknowns leading to longer cycles, greater costs, and more oversight.
Avoid Technology Overreach
Study after study notes that the DoD reaches for capabilities that are too far off—pushing for “exquisite” solutions as former Secretary of Defense Gates called them. These capabilities are often technology-driven and far beyond current reach. The GAO reports too many programs enter into one phase or another without mature technologies (GAO, 2011). As mentioned earlier in the Empirical Results section, immature technology at both the subsystem and system levels leads to delayed delivery of a promised capability to the warfighter, and to budget-busting cost overruns. The longer the program runs, the greater the temptation to add to it the latest technology or some other requirement, placing the program in what Spinney called a “death spiral development” (Fallow, 2002). A more disciplined approach to selecting the right technologies for a proposed system will enable program managers to have more effective control over schedules and, therefore, cost (Ward, 2010).
Keep It Simple
Complexity is also a factor that drives long acquisition cycles to push for increased performance at the expense of the schedule. It might be prudent to look at less complex systems that are not “silver bullet systems” capable of being all things to all users, but instead incorporate “ready-for-prime-time” technology. For example, the F-22 was jokingly referred to as the E/F/A/B/C/K-22 to indicate the DoD was counting on one aircraft to meet all its needs. Clearly the DoD does not have to defeat the adversary with a single system and should make its acquisition decisions with simplicity in mind.
Complexity is the antithesis of affordable, on-time systems. Dan Ward, in particular, has been writing on this topic for some time (Ward, 2012). As discussed in Department of Defense Instruction (DoDI) 5000.02 and the Better Buying Power initiatives promulgated by Under Secretary of Defense for Acquisition, Technology and Logistics Frank Kendall, a proven successful approach has been taking programs in bite-sized increments, shooting for the “80 percent solution” with evolutionary systems to follow. It may be far better to take a less capable, mature system now and build up to the full capability through evolutionary or block development. The F-16 is a notable example of incremental development, taking the “best” that could be made quickly for a reasonable cost, but adjusting to new technologies and adapting for the current challenges and operational experience (Quadrennial Defense Review Independent Panel, 2010). This reinforces the need to set realistic goals to meet the schedule.
Starting more programs than the Service can afford creates inefficiencies by lengthening programs. Inevitably, more programs will be competing for the same limited funds, thereby creating a slow, vicious death spiral cycle due to sparse budgets
Initiate Fewer Programs
Although DoDI 5000.02 stipulates full funding as an entrance requirement for a development effort’s individual phases, the DoD starts more programs than it can hope to fund through full production (Chaplain, 2011). This practice creates a myriad of problems—it lengthens acquisition cycles, creates pressure to underestimate costs, and eventually leads to quantity cuts that can precipitate a program’s death spiral. This results from the DoD and the Services pursuing perceived higher performance through a more robust portfolio of capabilities. However, more often than not, this impulse is driven by a lack of clear priorities on which capabilities or systems should be developed. Indeed, the need for better prioritization has been tacitly acknowledged through the recent revamping of the Joint Capabilities Integration and Development System. A consistent and analytically based prioritization of systems that can close the maximum number of capability gaps could encourage a reduction of the chaotic competition that plagues the defense acquisition system right now. For example, the DoD could adopt an algorithm similar to one RAND developed for prioritizing systems early in the acquisition cycle (Chow, Silberglitt, & Hiromoto, 2009).
Starting more programs than the Service can afford creates inefficiencies by lengthening programs. Inevitably, more programs will be competing for the same limited funds, thereby creating a slow, vicious death spiral cycle due to sparse budgets. Eventually, the DoD needs to admit some of the future systems are just not going to happen without huge infusions of cash, which seems unlikely. Banking on sufficient funds to cover all programs through full production places other systems in jeopardy, and while the investments may not be wasted completely, the money could be better spent elsewhere to produce on-time programs.
Previous research has failed to develop a theoretical framework from which to analyze tradeoffs within the acquisition system. Although the acquisition community has produced many potentially useful observations and recommendations since the early 1990s, this gap in the literature has arguably marginalized efforts to implement worthwhile policy changes. By making it seem as if the recommended policy changes were not motivated by any single overarching guiding principle, the impetus for a paradigm shift was weakened, and piecemeal changes and stalls were the result. This research effort seeks to unite many of the previous recommendations under a single theoretical rubric.
While this article has taken on a distinctly mathematical tone, the primary objective has been to develop an internally consistent framework for the values DoD leaders should communicate to the acquisition community. That is, the complexity of the defense acquisition system precludes resolution of priorities through a simple mathematical programming problem, but the insight gained through the inherent prioritization represented in the model allows for clarity of purpose. For the process to be disciplined, senior leaders must transmit unambiguous values through their statements and actions.
In keeping with this framework, the popular DoD convention of focusing primarily on costs should be eschewed, and more attention should be paid to meeting the schedule. Empirical results have demonstrated that cost performance is likely to improve through adoption of a new mindset, and the more realistic expectations concerning a system’s performance that are implied by this conceptual model will undoubtedly improve both cost and schedule outcomes.
To print a PDF version of this article, click here.
Dr. Chad Dacus is a defense analyst-economist at the Air Force Research Institute (AFRI). Before joining the AFRI staff, he worked for the Center for Naval Analyses (CNA) as both a field representative at U.S. Fleet Forces Command and as a research analyst at CNA headquarters. Dr. Dacus holds a PhD in Economics from Rice University and an MS in Statistics from Texas A&M University.
(E-mail address: email@example.com)
Col Stephen Hagel, USAF (Ret.), is a defense analyst at the AFRI. His current research interests include defense logistics, acquisition, and strategic planning. Prior to joining AFRI, he served on the Air War College faculty in the Warfighting Department. Col Hagel is a retired Air Force colonel. In the private sector, he was also employed as a reliability analyst and process improvement specialist at FedEx. Col Hagel holds an MS in Logistics from the Air Force Institute of Technology.
(E-mail address: firstname.lastname@example.org)
Chaplain, C. T. (2011). Space acquisitions: DoD delivering new generations of satellites, but space system acquisition challenges remain (Report No. GAO-11-590T). Testimony before the Subcommittee on Strategic Forces, Committee on Armed Services, U.S. Senate. Washington, DC: Government Accountability Office.
Chow, B., Silberglitt, R., & Hiromoto, S. (2009). Toward affordable systems: Portfolio analysis and management for Army science and technology programs. Santa Monica, CA: RAND Corporation.
Conrow, E. H. (1995). Some long-term issues and impediments affecting military acquisition reform. Acquisition Review Quarterly, Summer 1995, 199–212.
Dacus, C. (2012). Improving acquisition outcomes through simple system technology readiness metrics. Defense Acquisition Research Journal, 19(4), 444–461.
Deloitte. (2009). Can we afford our own future? Why A&D programs are late and over-budget—and what can be done to fix the problem. New York, NY: Deloitte, LLP.
Department of Defense. (2006). Risk management guide for DoD acquisition, sixth edition. Washington, DC: Office of the Under Secretary of Defense (Acquisition, Technology & Logistics).
Ewing, P. (2011, April 18). Your program has breached Nunn-McCurdy? Yawn. DoD Buzz [Online journal]. Retrieved from http://www.dodbuzz.com/2011/04/18/your-program-has-breached-nunn-mccurdy-yawn/
Fallow, J. (2002, June). Uncle Sam buys an airplane. The Atlantic Monthly, 289(6), 62.
General Services Administration. (2005). Federal Acquisition Regulation (FAR). Retrieved from http://www.acquisition.gov/far/current/pdf/FAR.pdf
Government Accountability Office. (2008). Defense acquisitions: Assessments of selected weapon programs (Report No. GAO-08-467SP). Washington, DC: Author.
Government Accountability Office. (2011). Defense acquisitions: Assessments of selected weapon programs (Report No. GAO-11-233SP). Washington, DC: Author.
Hoffman, M. (2010, August 16). Outgoing U.S. ISR chief slams slow acquisition. Defense News, p. 28.
Kadish, R. E. (2006). Defense acquisition performance assessment report. Washington, DC: Office of the Deputy Secretary of Defense.
Kendall, F. (2012). Better buying power 2.0: Continuing the pursuit for great efficiency and productivity in defense spending [Memorandum]. Washington, DC: Office of the Under Secretary of Defense (Acquisition, Technology & Logistics).
Quadrennial Defense Review Independent Panel. (2010). The QDR in perspective: Meeting America’s national security needs in the 21st century. The Final Report of the Quadrennial Review Independent Panel. Washington, DC: United States Institute of Peace.
Thompson, M., & Karon, T. (2009, April 6). Gates proposes big shift in Pentagon spending. Time. Retrieved from http://content.time.com/time/nation/article/0,8599,1889673,00.html
Ward, D. (2010, March-April). Faster, better, cheaper revisited: Program management lessons from NASA. Defense AT&L, 213(2), 48–52.
Ward, D. (2012, April). Acquisition reform: For real. Armed Forces Journal.