To print a PDF copy of this article, click here.
Congratulations! Since you wrote code in the past, you’re now designated as a software program manager for automated information systems (AISs) and information technology (IT). Don’t forget, you developed embedded digital engine control code or perhaps published vehicle dynamics modeling software, and so human resources now deems you as “in-the-know” about all matters IT, AIS and/or Defense Business Systems (DBS) technology. You have now been assigned to start managing one of the Department of Defense (DoD) IT/AIS programs somewhere in the system’s engineering process—perhaps in requirements or functional analysis and allocation or in synthesis.
During the 1990s’ dot.com boom, and continuing in today’s “post-personal computer era,” the DoD has had trouble retaining cyber experts due to the lure of the private sector. Since losses are unlikely to be stanched anytime soon, a great deal of technically savvy, but not IT-specialized, folks are being shunted into IT/AIS/DBS program management. This happens because the domain of science and technology (S&T), which includes AIS/IT, is not well understood by many decision makers. “The needs of the Service” prevail, which raises the question of what hardware-centric acquisitions experts need to unlearn to avoid unwittingly injecting cost, schedule or capability slippage into their programs. Well, it’s time to learn quickly that AIS/IT/DBS and software have some important fundamental differences where your experience can lead you astray. So what are the top things you need to unlearn? Here are some lessons learned the hard way:
- In scheduling out your program, realize software tech state-of-the-art is blazingly fast-paced. For example, one generation of gas turbine technology development encompasses almost 10 generations of software development and three to four generations of AIS/IT hardware. Fourth-generation fighters like the F-16 and F-15 have been around for 40 years and finally were eclipsed about a decade ago. In that same period, IT hardware evolved from minicomputers (PDP-8), through 8- to 64-bit personal computers, single-core to eight-core, and onward to the handheld device. When your system is being designed, keep a wary eye on not only the hardware obsolescence but also that of the software components. Press hard for mitigation strategies and a loosely coupled architecture. Remember when that F-16 fleet was just nearing initial operating capability back in the early 1980s? How much luck will you have opening on today’s nonclassified network computer the Fielding Plan that was written in Word Perfect for DOS v5.0? Similarly, will your mission-critical database migrate across the iron to new operating systems? Be a futurist and think through on what data standards for exchange, formatting and transmission this future event will rely? Get familiar with the Joint Capabilities Integration and Development System ITBox process if you’re working requirements.
- Recognize that software configuration management is perhaps even more critical than that for hardware systems. There is a compulsion to keep tweaking code, thereby succumbing to requirements creep and “gold plating” with the attendant risk of completely losing configuration control. This is due to the perceived malleability of code. The key word here is “perceived,” because tracking software changes and their introduction of second-order effects can be more tedious than actually making the changes. It is perhaps telling that Linus Torvalds, founder of the Linux operating system, seems to have felt that his greater contribution was the source version control system Git, which was developed to track versions of and allow scaling up his first contribution. It is also revealing that the Capability Maturity Model Integration concentrates more on software management than on the software product itself.
When costing unit production costs in Engineering, Manufacturing and Design, it is best to dump your hardware-centric thinking. Once code is written, debugged, passed through Developmental and Operational testing and the first compact disc is pressed, the unit cost to scale up is minuscule. The rare exception is software components that are commercial-off-the shelf (COTS) items for which per-processor and/or annual licensing, and/or software as a service costs may apply. By the way, direct licensing costs and avoidance of the recurring management burden to deal with them, not to mention baked-in data rights, are excellent reasons to explore the 2009 DoD Chief Information Officer memorandum on (free and) open source software to be deemed a commercially viable industry competitor. Know that hardware components are nearly always COTS and that a full technical data package may be hard to source. You may be further constrained by DoD IT equipment and software enterprise buys for many of your components. And while grousing about this loss of agility, admit that it does have an upside, such as leveraging enterprise bulk buys and helping to ensure parts traceability back to the foundry (per Open Trusted Technology Provider Standard of the International Organization for Standardization and the International Electrotechnical Commission). The latter benefit is not to be underestimated in this age of pressing cybersecurity concerns.
- To comply with security, safety and privacy imperatives for Command, Control, Communications, Computers and Intelligence systems, weapon systems, and DBS, respectively, in this post StuxNet world, the aforementioned supply chain integrity is important for software, firmware and hardware. It’s not just Windows 10 being the “bad boy” phoning home as mentioned in the lay press; Cisco routers have been found with “backdoor” code in their firmware that presents potential opportunities for espionage, or worse, sabotage. As a non-cyber acquisitions subject-matter expert, recognize that if it’s on the DoD Information Network—and even if it’s not—but merely executes binary code (e.g., “pushes 1’s and 0’s”), that, by definition, it is not behind the base fence but is out there in the public square and vulnerable to attack. From Day One of your program’s architecting within DoD Architecture Framework Version 2, cybersecurity needs to be baked into your design and not bolted on. In a similar vein, net-centricity and utilization of open standards are critical capabilities and provide a major hedge against obsolescence. So while at, dust off that copy of DoD Instruction 5000.02 (Operation of the Defense Acquisition System) and give it another read through; this time dwell heavily on Enclosures 11 and 12, which are focused on pressing topics in defense software systems acquisition.
- Unlike hardware components (gears, pistons and bearings), defense information and software systems do not fail on a Weibull or “Bathtub” curve. Outside of a select few components, like muffin fans, hard disk drive bearings and switching power supply transistors, bits and bytes do not wear out with duty cycling over their service life. Software stack components do undergo a high-velocity of capability upgrades and bug fixes, thanks yet again to that malleable nature of software. Data exchange standards evolve, application programming interfaces morph, and portions of the software stack get patched and modernized and so introduce second- and third-order changes. This leads to the fifth point and that is …
- Recognize that data rights are as critical, if not even more so, than protected rights in hardware systems. Reverse engineering by software decompilation often is prohibited by End User License Agreements (EULAs), is a more arcane skillset, and often yields cryptic results. Know that the ultimate technical documentation in the software world includes, but is not limited to, well-commented and -structured code in a vendor-neutral language like ANSI C, Fortran77 (as opposed to, say, VBasic or Oracle Java). Recognize that the whole system stack, from the bare metal hardware up to your end-user application, may impact your system’s reliability and maintainability, even its ability to function. It does no good to have a vendor write a VBA-based solution when your infrastructure is to be run on a Portable Operating System Interface-compliant operating system like Linux. Given the massively interconnected, constant operation of many DoD software-intensive and DBS systems, interface control (now central to your form, fit, function and interface [F3I] thinking) to standards are paramount, leverage them!
Now your arrival from the hardware-centric world does not totally disadvantage you; you bring some humility into software systems acquisition with that general lack of knowledge and therefore lack of institutional inertia. You are primed to foresee things those who have grown up within the AIS/IT world often totally miss or assume away:
First, unlike your IT brethren, you realize software often is the long pole in the tent for major systems schedule and technical risk; you may well have directly experienced this in previous tenure. I certainly did: Unlike many born-in-the-DoD program managers, I was a performer-integrator. In the late 1990s, the Office of Naval Research commissioned a deep-ocean intervention robotic submarine. The basic hardware of the vehicle—ballast, pressure vessels, sensors, fairing, thrusters, power distribution and major computing systems hardware, among other things—were all ready within 2 years of program kickoff. But realizing all the proposed capabilities in the software portion of this effort (SAUVIM) took yet another 3 to 4 years.
Second, your AIS/IT brethren often lack configuration management discipline, but you are sold on it. Let’s face it, it’s hard to change things around once you’ve “cut metal,” and there is much lead time in sourcing extra material, tooling and skilled manpower. Meanwhile, the software developer’s lexicon is salted with “sprints,” “scrums,” “jams” and “rapid spirals”; this is indicative of a Red Bull-fueled, Wild West mentality. And while it may lead to the next killer application like Angry Birds or Facebook, it can also doom a project for which the stakes on configuration are a little higher due to much more massive integration requirements, not to mention differing consequences for failure. Know that software program complexity does not scale linearly with project size; figure it to be more exponential in nature.
Third, you possess a holistic life-cycle view of programs from the outset since you come from a world where systems and components wear out, and so you already think in terms of bathtub curves, ancillary equipment, facilities, maintenance documentation and spares provisioning—perhaps because items are more tangible. Software program managers often neglect to plan provisioning for compilers, development environments, documentation and long-term interoperability; you can help save them from neglecting these life-cycle issues. While IT-pedigreed folks are accustomed to everything being “COTS-on-a-warranty,” you can see beyond this paradigm and are not blind to other options with their life-cycle cost implications. Your IT brethren may blindly accept yearly software licensing burdens as “the cost of doing business.” Your hard-won hardware experience may see a more optimal solution. Is the best plan buying government-off-the-shelf with well-commented code or should you look at COTS code, or even a free open source software ([F]OSS)-based solution? Is the 3-year warranted blade server iron in-house running GOTS software truly the best solution or would sourcing an accredited infrastructure- or platform-as-a-service (IaaS, PaaS) contract better meet the requirements with enhanced capabilities and feature a cheaper life cycle to boot?
Continue to assert your data rights with vigor as in this systems realm they are even more at risk due to your colleague’s easy acquiescence to “it’s always done this way” (a corollary of “You can’t go wrong buying Microsoft/Oracle/Novell/etc.!”), the rarity of skill needed to reverse engineer compiled codes, and the statutory hooks that COTS software vendors load into their EULAs.
IT folks have a culture of doing it in-house, as a material systems expert that you know to engage industry and academia early and often to keep tabs on the state-of-the-art and best practices. And for this fast-moving area, do not skip engaging these folks for the informal market survey and the more formal analysis of alternatives, even for a low-dollar-value program.
You’ve got homework and reading lists ahead, but as the able science, technology, engineering and mathematics person who is a newbie to the world of DoD IT intensive program management, where do you start? It would be hard to begin with a short list. But to bootstrap your thinking across such diverse topics as architecting, cybersecurity and recent historical developments in the cloud consider, respectively: Barry Boehm, Peter Kind and Richard Turner’s article “Risky Business: 7 Myths about Software Engineering that Impact Defense Acquisitions,” in the May-June 2002 issue of the Defense Acquisition University’s Program Manager; Kim Zetter’s “An Unprecedented Look at StuxNet, the World’s First Digital Weapon” published in Wired on Nov. 3, 2014; and, if you get a chance, Gartner Vice President and analyst Doug Laney’s Gartner Symposium presentation “55 Examples of Big Data Case Studies in 55 Minutes.”
Get to know the nuances of the following terms via a little primary school-style vocabulary drill: seven-layer OSI model, virtualization, datacenter, Internet Protocol Version 6, IaaS/PaaS/SaaS, the internet of things (IoT), net-centricity, asymmetric-key, Big Data and cloud computing. Most of all, do a little refresher “Hello World” programming in code to familiarize yourself with the software creation process. May I suggest Brian Kernighan and Dennis M. Ritchie’s book The C Programming Language as very good exercise for the new program manager or systems engineer?
In closing, I also mention that the Defense Acquisition University itself has some very helpful short course modules to help with initially getting up to speed. Yes, you may be the newbie in the room, but at the same time realize you also bring a very valuable outsider’s viewpoint and humility to this world. The DoD really needs this perspective given the 26 percent “success ratio” in software intensive systems, with the DoD managing only 18 percent (and 0 percent once above a $10 million level-of-effort) as cited upfront in the Boehm-Kind-Turner article.
Easterday is deputy branch chief of the Sustainment Branch at the Air Operations Center, U.S. Air Force C2 Requirements Division, Headquarters Air Combat Command, in Hampton, Virginia. He is an Air Force developmental engineer with 7 years of experience in turbine engine Science and Testing development and 4 years in depot sustainment of airframe line replaceable units.
The author can be contacted at firstname.lastname@example.org.