Category Archives: ARJ 71 | October 2014

ARJ 71 | October 2014


To print a PDF copy of the complete issue, click here. Individual articles can be printed within the articles themselves.


To print a PDF copy of the complete issue, click here. Individual articles can be printed within the articles themselves.

From the Chairman and Executive Editor

Dr. Larrie Ferreiro, Executive Editor, Defense ARJ

Chairman Larrie Ferreiro

The theme for this edition of Defense Acquisition Research Journal is “…All Others Must Bring Data.” It derives from the famous quote by American management consultant W. Edwards Deming, “In God we trust; all others must bring data,” displayed outside the office of Under Secretary of Defense for Acquisition, Technology, and Logistics Frank Kendall. Accurate and meaningful data are the basis for making informed acquisition policy decisions; as British scientist William Thomson (Lord Kelvin) said over a century ago, “When you can measure what you are speaking about … you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.”*

Continue reading

Compressing Test and Evaluation by Using Flow Data for Scalable Network Traffic Analysis


To print a PDF version of this article, click here.

Authors: Kevin Buell, Mustafa G. Baydogan, Burhan Senturk, and James P. Kerr

The specialized nature of technology-based programs creates volumes of data on a magnitude never before seen, complicating the test and evaluation phase of acquisition. This article provides a practical solution for reducing network traffic analysis data while expediting test and evaluation. From small lab testing to full integration test events, quality of service and other key metrics of military systems and networks are evaluated. Network data captured in standard flow formats enable scalable approaches for producing network traffic analyses. Because of its compact representation of network traffic, flow data naturally scale well. Some analyses require deep packet inspection, but many can be calculated/approximated quickly with flow data, including quality-of-service metrics like completion rate and speed of service.

Continue reading

Improving Statistical Rigor in Defense Test and Evaluation: Use of Tolerance Intervals in Designed Experiments


To print a copy of this article as a PDF, click here.

Author: Alethea Rucker

Leveraging the use of statistical methods is critical in providing defensible test data to the Department of Defense Test and Evaluation (T&E) enterprise. This article investigates statistical tolerance intervals in designed experiments for the T&E technical community. Tolerance intervals are scarcely discussed in extant literature as compared to confidence/prediction intervals. The lesser known tolerance intervals can ensure a proportion of the population is captured in the design space, and have the ability to map the design space where factors can be reliably tested. Further, the article investigates several two-sided approximate tolerance factors estimated by Monte Carlo simulation and compares them to the exact method. Finally, the applicability of tolerance intervals to the defense T&E community is presented using a simple case study.

Continue reading

A Comparative Analysis of the Value of Technology Readiness Assessments


To print a PDF version of this article, click here.

Authors: Reginald U. Bailey, Thomas A. Mazzuchi, Shahram Sarkani, and David F. Rico

The U.S. Department of Defense endorsed and later mandated the use of Technology Readiness Assessments (TRAs) and knowledge-based practices in the early 2000s for use as a tool in the management of program acquisition risk. Unfortunately, implementing TRAs can be costly, especially when programs include knowledge-based practices such as prototyping, performance specifications, test plans, and technology maturity plans. What is the economic impact of these TRA practices on the past and present acquisition performance of the U.S. Army, Navy, and Air Force? The conundrum today is that no commonly accepted approach is in use to determine the economic value of TRAs. This article provides a model for the valuation of TRAs in assessing the risk of technical maturity.

Continue reading

Where Are the People? The Human Viewpoint Approach for Architecting and Acquisition


To print a PDF version of this article, click here.

Authors: Holly A. H. Handley and Beverly G. Knapp

The U.S. Department of Defense Architecture Framework (DoDAF) provides a standard framework for transforming systems concepts into a consistent set of products containing the elements and relationships required to represent a complex operational system. However, without a human perspective, the current DoDAF does not account for the human performance aspects needed to calculate the human contribution to system effectiveness and cost. The Human Viewpoint gives systems engineers additional tools to integrate human considerations into systems development by facilitating identification and collection of human-focused data. It provides a way to include Human Systems Integration (HSI) constructs into mainstream acquisition and systems engineering processes by promoting early, frequent coordination of analysis efforts by both the systems engineering and HSI communities.

Continue reading

Book Review: Adapting to Flexible Response, 1960–1968


To print a PDF version of this article, click here.

book-cover-adaptingSeries: History of Acquisition in the Department of Defense, Volume II

Author: Walter S. Poole

Publisher: Office of the Secretary of Defense, Historical Office

Copyright Date: 2013

ISBN: 978-0160921834

Hard/Softcover: Hardcover, 467 pages

Reviewed by: Dr. Roy L. Wood, Dean, Defense Systems Management College, Defense Acquisition University
Continue reading