To print a PDF copy of this article, click here.
In the May-June 2013 issue of Defense AT&L magazine, Under Secretary of Defense for Acquisition, Technology and Logistics Frank Kendall authored an article titled “The Original Better Buying Power—David Packard Acquisition Rules 1971.” Packard’s fifth acquisition rule was “fly before you buy”—the underpinnings of test and evaluation (T&E). I thought it might be interesting to look at another challenge that then Deputy Secretary of Defense David Packard confronted in 1971: what to do about test and evaluation in the Department of Defense (DoD). The original Director for Test and Evaluation created by David Packard was a DT&E with broad responsibilities for all T&E matters in the DoD. Today’s DT&E is focused only on Developmental Test and Evaluation. This article provides a brief snapshot into the challenges involved in forming the first DT&E office in the Pentagon and its evolution into the DT&E office today.
In July 1970, President Nixon’s Blue Ribbon Defense Panel (BRDP) reported its findings. The “Fitzhugh Commission,” as it is sometimes called, in recognition of its chairman, Gilbert W. Fitzhugh, had been appointed 1 year earlier with the broad charter to report and make recommendations on the organization and management of the DoD, its research and development efforts, and its procurement policies and practices. The final report prominently featured a discussion of operational test and evaluation, and the panel’s recommendations had significant influence on how David Packard would shape the role of T&E to support defense acquisition. (The BRDP is available at http://www.dtic.mil/dtic/tr/fulltext/u2/a013261.pdf .)
The BRDP had three main findings relative to the conduct of T&E: “functional testing” (we use the term “developmental testing” today) is fundamentally sound; operational testing is generally inadequate, and the Services do not conduct enough joint test and evaluation. Regarding “functional” testing, the BRDP wrote:
By and large, functional testing in and for the Department of Defense appears to be well understood and faithfully executed. Serious policy deficiencies are not apparent, and such failures in functional testing as occur can be primarily attributed to lack of technical competence, oversight, or procedural breakdowns. Functional testing is not considered to be a major problem area.
Pertaining to operational test and evaluation, however, the BRDP found that “Operational test and evaluation has been too infrequent, poorly designed and executed, and generally inadequate.” The panel’s recommendations regarding T&E, and thus the actions taken by DoD to implement those recommendations, essentially focused on correcting deficiencies in operational test and evaluation (OT&E), and have driven the primary focus in defense acquisition for the past 40 years.
Prior to 1976, the fiscal year began on July 1 and ended on June 30; in other words, Packard gave the DoD just 4 months to put its house in order!
The BRDP also highlighted the lack of OT&E oversight in OSD as a “glaring deficiency” and reported that “In connection with test and evaluation, it should be emphasized that responsibilities for any evaluation function must be exercised independently. When they are subordinated to or combined with responsibilities for the development of the item or subject being evaluated, the requisite objectivity is seriously jeopardized.” The BRDP also considered evaluation within a much broader construct and recommended that the Secretary create a position of “Deputy Secretary of Defense for Evaluation” supported by three Assistant Secretaries: Comptroller, Program and Force Analysis, and Test and Evaluation. The BRDP further recommended that the Secretary create a Defense Test Agency “to perform the functions of overview of all Defense test and evaluation, designing or reviewing of designs for test, monitoring and evaluation of the entire Defense test program, and conducting tests and evaluations as required, with particular emphasis on operational testing, and on systems and equipments which span Service lines.”
Packard promptly began acting on the BRDP recommendations. In the first 8 months of 1971, he signed three memoranda that made sweeping changes to the role of test and evaluation in support of defense acquisition.
In the first memorandum, “Conduct of Operational Test and Evaluation,” dated Feb. 11, 1971, Packard wrote, “… a number of specific actions must be taken to put our house in order in this very important aspect of the weapon system acquisition process.” Packard directed three specific actions:
- Services will restructure their organizations for OT&E to be “separate and distinct from the developing command” and report the results of its test and evaluation efforts “directly to the Chief of the Service.”
- OSD will establish “a Deputy Director for Test and Evaluation within the Office of Director of Defense Research and Engineering (ODDR&E) with across-the-board responsibilities for OSD in test and evaluation matters. This office will review and approve test and evaluation plans prepared by the Services and will provide an assessment of results obtained.”
- Services will conduct more joint operational test and evaluation.
In the Feb. 11, 1971, memo, Packard asked the Service Secretaries to advise him of their plans before March 31, and to “be prepared to have the new structure in effect by the end of this fiscal year.” Prior to 1976, the fiscal year began on July 1 and ended on June 30; in other words, Packard gave the DoD just 4 months to put its house in order! Clearly, this was a high priority for the Deputy Secretary.
The second of Packard’s memoranda, dated April 21, 1971, cemented the role of the Deputy Director for T&E in the Defense System Acquisition Review Council (DSARC) (the DSARC was the forerunner of the modern Defense Acquisition Board). Packard directed the Military Departments to provide their assessment of test results to the DDR&E prior to the DSARC Milestone III production decision. The Deputy Director for Test and Evaluation (DDT&E) would review those results and provide his independent recommendations at the Milestone III meeting.
I have come to believe that the department’s decades-long emphasis on OT&E in the acquisition life cycle took our eye off the target.
Packard also quickly began honing the new processes as they were implemented. On Aug. 3, 1971, Packard wrote, “Based on experience to date resulting from [the previous two memos], I consider certain further clarification and instruction is required.” In this third memorandum, “Test and Evaluation in System Acquisition Process,” Packard described in more detail requirements for conduct of adequate OT&E. Additionally, Packard assigned these additional responsibilities to the Deputy Director for Test and Evaluation:
- “be responsible within OSD for reviewing test and evaluation policies and procedures and recommending to me changes as appropriate;
- “monitor closely test and evaluation conducted by the Services for DSARC programs, and such other programs as he believes necessary, throughout the entire testing cycle;
- “report to the DSARC and directly to me at DSARC Milestones I and II his assessment as to the adequacy of the list of critical issues and problems to be attacked by test and evaluation and the schedule of test milestones;
- and “report at Milestone III to the DSARC and directly to me his independent recommendation.”
The Deputy Secretary went on to require that “Service test plans and test results be made available at his request as early as developed.” Additionally, the Deputy Director for Test and Evaluation would be responsible for “initiating and coordinating appropriate joint testing; overseeing the test and evaluation of foreign systems for possible DoD use; and administering for OSD its responsibilities for the national and major Service test ranges.”
Packard needed someone to lead the new organization and make his vision reality. On June 7, 1971, Packard appointed retired Lt. Gen. Alfred Dodd Starbird to be the first Deputy Director for Test and Evaluation. Lt. Gen. Starbird set out to build the new DDT&E organization. He had three assistant directors with responsibilities for Tactical Systems T&E, Strategic and Support Systems T&E, and Test Resources. During his tenure, Starbird recommended the Defense Science Board (DSB) examine test and evaluation in the department. The DSB published the first report on T&E in April 1974; it would be the first of six reports the DSB would issue on T&E over the next 40 years. Starbird also initiated the publication of the first DoD Directive on test and evaluation: DoD Directive 5000.3, Test and Evaluation, was published on Jan. 19, 1973. The 5000.3 continued in effect, with four updates, until 1991, when the February 1991 issuance of DoD Directive 5000.1 canceled the 5000.3 and T&E became Part 8 of DoD Instruction 5000.2. Starbird also was able to insert into the defense budget a new appropriation for the deputy director of T&E. The Department of Defense Appropriation Act for FY 1973, Public Law 92-570, dated Oct. 26, 1972, included the following language:
DIRECTOR OF TEST AND EVALUATION, DEFENSE
For expenses, not otherwise provided for, of independent activities of the Director of Defense Test and Evaluation in the direction and supervision of test and evaluation, including initial operational testing and evaluation; and performance of joint testing and evaluation; and administrative expenses in connection therewith, $27,000,000, to remain available for obligation until June 30,1974.
Thus did David Packard implement the recommendations of the BRDP. Although there would not be a Deputy Secretary for Evaluation, or an Assistant Secretary for T&E, he did create the “defense test agency” in the form of the Office of the Deputy Director for Test and Evaluation in the Office of the Director of Defense Research and Engineering. The Deputy Director for T&E was responsible for all matters involving T&E almost exactly as the BRDP recommended. Importantly, Packard also ensured the Deputy Director for T&E provided his independent assessment at the DSARC. However, independence came to be another issue entirely, and the T&E organization Packard put in place would not endure.
Concerns about OT&E continued to consume attention in the Pentagon and especially on the Hill. Likewise, there was an overwhelming sense that assignment of the Deputy Director for Test and Evaluation under the DDR&E posed a conflict of interest. The DDR&E was the DoD’s chief acquisition official at the time, which violated the BRDP’s premise that when responsibilities for evaluation “are subordinated to or combined with responsibilities for the development of the item or subject being evaluated, the requisite objectivity is seriously jeopardized.” I doubt that Starbird, or any of the deputies after him, felt that way, but regardless, in September 1983, Congress established a presidentially appointed, Senate-confirmed position of Director, Operational Test and Evaluation, independent of the acquisition authority and reporting directly to the Secretary of Defense. The position of Deputy Director for Test and Evaluation was renamed Deputy Director for Developmental Test and Evaluation. Another 16 years later, on June 7, 1999—or 28 years after Packard created it—Secretary of Defense William Cohen disestablished the test office within what had become the Office of the Undersecretary of Defense for Acquisition and Technology. During those nearly 3 decades, all emphasis in T&E in the department continued to be on operational testing, and Cohen’s decision was intended specifically to strengthen the Office of the DOT&E; however, it virtually eliminated oversight of developmental test and evaluation. Congress would reverse this 10 years later. How that came to pass is another story.
As a member of the DT&E organization in OSD today, my view of test and evaluation is obviously biased. But with 15 years’ experience in the T&E community, most of which has involved operational testing in some capacity, I have come to believe that the department’s decades-long emphasis on OT&E in the acquisition life cycle took our eye off the target. Acquisition success is not about passing OT&E and getting that full-rate production decision; success occurs when we have properly set the conditions to begin production. The key to improving acquisition outcomes is to get the development right, and verify it through rigorous developmental test and evaluation. We have to do a better job helping programs get to Milestone C; that’s the target we must aim for. In T&E, we have to shift left if we are going to “fly before you buy.” Packard had a good plan back in 1971, but we lost track of the target. Given his unprecedented (and unmatched) level of involvement in shaping the course for T&E in the department, I think one could make a convincing argument that David Packard was, himself, the original DT&E.