Application of System and Integration Readiness Levels to Department of Defense Research and Development

To print a PDF copy of this article, click here.

Author: Sean Ross

Technology Readiness Level only tells part of the story of system maturation. As component technologies are developed to become part of systems, there are also integration and manufacturing issues to consider. This article improves upon the System and Integration Readiness Level concepts previously developed by B. J. Sauser et al., combines the concepts of Technology, Integration, and Manufacturing Readiness Levels, adapted for use in defense acquisition, into a single metric—System Readiness Level. This metric can then be used as an indicator to identify areas for resource allocation to enable the most efficient path to technology transition and to prevent premature system advancement.

In an ideal world, a component technology would develop concurrently with its interfaces and its ability to be manufactured. In the real world, technologies lead both their interfaces and manufacturing infrastructure. For example, motorcycles were first made with fixed foot-pegs until some rather spectacular, spin-out wrecks occurred, prompting folding foot-pegs. The human-motorcycle interface maturity followed the technical maturity at the expense of safety.

arj-78-article-1-secondary-2Early airplanes were made, one-at-a-time, with bicycle manufacturing equipment. The manufacturing maturity lagged the technology. The competing pitfall in system development is the premature advancement of a technology to the next level of development in advance of its interfaces, such as the current state of the F-35 program. Although the program is in late stage development, interface and component technology issues are still emerging that are preventing full operational capability (Bender, 2015).  We can do a better job by minimizing the gap between interface, manufacturing, and technology maturity. Integration and system readiness are not yet implemented in any formal way Department of Defense (DoD)-wide.

This article explains a method to combine Technology Readiness Level (TRL) (See Appendix, Table A-1), Integration Readiness Level (IRL), and Manufacturing Readiness Level (MRL) (See Appendix, Table A-2) into a single metric—System Readiness Level (SRL)—that can provide guidance to decision makers during the technology maturation process. Such guidance can minimize the delays and mishaps likely to occur when interfaces and manufacturing significantly lag their component technologies.


arj-78-article-1-secondary-3The DoD Research, Development, Test and Evaluation budget is subdivided into seven separate activities: basic research; applied research; advanced technology development; advanced component development and prototypes; system development and demonstration; research, development, test and evaluation (RDT&E) management support; and operational systems development, i.e., the DoD categories of funding and technology development (Appendix, Table A-3). These seven activities are designated as DoD 6.1 through 6.7. This article incorporates the 6.1 through 6.7 levels of funding and appropriate levels of maturity so that the same metric can be used throughout the acquisition life cycle. Verbal definitions of TRL, MRL, IRL, and SRL are included at the end of the article.

Sauser, Ramirez-Marquez, and Devanandham, and Dimarzio (2007), and Sauser, Ramirez-Marquez, Magnaye, and Tan (2008a) furthered the concepts of TRL to include IRL and SRL (Sauser, Forbes, Long, & McGrory, 2009; Sauser, Gove, Forbes, & Ramirez-Marquez, 2010). These approaches emphasize that the interfaces between subsystems are every bit as important as the subsystems themselves, and that no system can be deemed ready for deployment based on the component technologies alone.


Sauser’s basic approach is to imagine a system composed of component technologies from 1 to n, each with a TRL as shown in equation (1) and Figure 1.


Mathematical Note. A list of symbols or numbers in braces represents a vector. A subscripted symbol indicates one element out of a vector. A number without subscripts indicates the whole vector quantity. Lower case is used for normalized quantities.

Figure 1. A System as a Collection of Component Technologies


For example, a motorcycle can be viewed as an engine, power train, exhaust, electrical system, cooling system, saddle, suspension, wheels, gauges, steering, headlamp, etc.

Each component technology has a potential interface with each other component and with the external environment, including the possibility of an interface going both ways, as shown in equation (2). For simplicity, Figure 2 shows the interfaces with double arrows, as if irl12 = irl21, which need not be the case. IRL must be expressed as a two-dimensional matrix rather than a one-dimensional vector. The vector is generally square—with the same number of rows and columns. The diagonal of the matrix is not used since a technology always works with itself.


Figure 2. System as a Collection of Interfaces and Component Technologies

arj-78-article-1-figure-2In the Sauser approach, The IRL matrix and the TRL vector are multiplied together as a vector product (U.S. Navy, 2009, p. 35) to form an SRL vector that can be averaged for an overall SRL (Sauser, Verma, Ramirez-Marequez, Gove, 2006, p. A-12; Sauser et al., 2007, p. 681; U.S. Navy, 2009, p. 33). Note that this paper shows matrix notation in both reduced tensor notation and matrix notation as a convenience for a multidisciplinary audience. SRLj, [SRL] and SRL all refer to the same vector entity and all versions of equation (3) show the same tensor/matrix operation in different notation. Equation (4) shows the Sauser formula for SRL. Computational and practical examples of all formulas will be shown in the examples section.


As shown in equations (3a) and (3b), the Sauser mathematics views a component of SRL (SRL<sub”>i ) as being based upon a single interface type and its associated technologies; the SRL1 component includes TRL1, TRL2, etc., and all of the IRL1n rather than a technology-centric approach that included TRL1 with all its interfaces. The interface-centric approach is graphically shown in Figure 3 and contrasted with a technology-centric approach in Figure 4 using a motorcycle. The mechanical component of SRL (SRLmechanical) in the Sauser approach for a motorcycle would be based upon the mechanical-engine, mechanical-headlamps, mechanical-saddle, mechanical-tires, etc., interfaces. The interface-centric approach has some serious limitations as will be covered in the next sections.

Figure 3. A component of SRL based upon a single interface and its associated technologies


arj-78-article-1-figure-4The average of the SRL vector, equation (4), describes how mature the system is. The Sauser approach may make sense for a single mission or project, such as the deployment of a new software system. However, it has some serious drawbacks for use in research and development where planners need to decide what technologies to develop for the eventual deployment of a new platform, weapon, or system. First, SRL, as defined in the U.S. Navy’s Littoral Combat Ship Mission Module Program System Maturity Assessment Guide (2009), is interface-centric as opposed to component-centric. The Sauser definition shows each interface with its associated technologies rather than each technology with its associated interfaces. Second, SRL as defined by Sauser, has no clear meaning assigned to a given numerical value. In one presentation (Sauser, Ramirez-Marquez, Magnaye, & Tan, 2008b), SRL is defined along a value from 0 to 1 with five unequal intermediate levels and no verbal definitions akin to those for TRL, IRL, and MRL. This gives SRL a different kind of scale than IRL and TRL, which are clearly defined such that 1 is a concept and 9 is full deployment. Third, the Sauser-defined SRL only has meaning at the full system level. The interface-centric components of the SRL vector give no guidance to component developers. Finally, the definitions of IRL tend to be information technology (IT)-centric, emphasizing control and information. IRL needs to be applicable to a wide variety of interfaces, including mechanical, thermal, electrical, structural, and control interfaces as well as logistics, policy, and other ‘-ility’ and mission interfaces.


Characteristics of a Useful System Readiness Level Metric

A useful metric will be defined so as to give a clear indication for planning resource allocation. SRL and IRL, as metrics, can be useful if they are defined correctly. The author proposes the following criteria for a useful SRL and IRL metric.

  1. IRL definitions should be applicable to a wide variety of technologies.
  2. SRL should be defined such that SRL=1 is a concept and SRL=9 is a mature, deployed system on the same basic scale as TRL, MRL, and IRL.
  3. SRL should equal TRL when the interfaces are developed concurrently with the components, and should be less than TRL when interfaces are less mature than the components. This will give planners a clear metric that lets them know that it is time to transition funding into more interface-centric development or to proceed with component technology maturation.
  4. SRL should be technology- or component-centric, not interface-centric. This makes it clear when a particular subcomponent is not able to progress further toward implementation due to an interface or manufacturing issue.
  5. SRL should include MRL, TRL, and IRL.
  6. SRL should be applicable to a wide variety of technical maturities (see Appendix, Table A-3), including basic research (6.1 funding); applied research (6.2 funding); advanced technology development (6.3 funding); advanced component development and prototypes (6.4 funding); system development and demonstration (6.5 funding); and operational systems development (6.7 funding), i.e., the DoD categories of funding and technology development. Note that 6.6 funding is not included because it is for management activities and not tied to a level of technical maturity.
  7. SRL must be defined in such a way as to avoid maturity in one component overshadowing immaturity in another (Kujawski, 2010) and giving the illusion that the system is ready to progress. This implies that SRL should never be able to be greater than TRL at either the system or component level.

Proposed System Readiness Level Metric

The author proposes that a more useful way to arrange MRL, TRL, and IRL is as a series of normalized dot products, rather than vector products (Sauser et al., 2008a, p. 47). This changes the view of the components of SRL from being interface-centric to being technology-centric, as shown in the contrast between Figure 3 and Figure 5, and between the right and left sides of Figure 4. The SRL components are equal to the product of the normalized MRL, the TRL, and the mean of the normalized IRL, as shown in Table 1. In the notation that follows, upper case is reserved for standard (i.e., verbal) definitions and lower case is for normalized quantities. Note that the word ‘system’ in this article refers to a generic system—anything that can be usefully viewed as being composed of parts, rather than specifically as a deployed military asset.

Likewise, the term ‘component’ refers to the parts that make up a larger grouping rather than exclusively as a line-replaceable item with a specific part number. The term interface should be viewed in the broad sense of the word to also include the external environment—the ‘ilities’ (availability, maintainability, vulnerability, reliability, supportability, etc.) and the DOTmLPF-P (Doctrine, Organization, Training, materiel, Leadership and Education, Personnel, Facilities-Policy).



Table 1. Normalized Integration readiness level definitions


To have SRL equal to TRL when IRL and MRL are at commensurate levels of development requires normalized versions of IRL and MRL scaled to the level of research. Basic research (6.1 funding) should have a goal of an IRL of 1 (Interface identification) and MRL of 2 (Manufacturing concepts identified), so that the normalized mrl and irl equal 1 when the appropriate levels of IRL and MRL are reached. Likewise, system development and demonstration (6.5 funding) should have as its goal an IRL of 6 (interface control) and an MRL of 6 (prototype in a production-relevant environment) so the normalized mrl and irl equal 1 when the appropriate levels are reached. The signal to proceed to the next step in system development occurs when SRL equals TRL, indicating that the interfaces and manufacturing base are at a commensurate level of development with the component technologies. The nomenclature and definitions for normalized IRL are shown in Table 1. The normalization factors are chosen to be consistent with the funding categories listed in the DoD Research, Development, Test and Evaluation (RDT&E) budget (Appendix, Table A-3). Different communities may have differing levels of MRL, TRL, and IRL goals vs. acquisition stage so that the normalization factors are intended as starting suggestions. It would also be viable to have normalization factors based on the DoD 5000.02 Model 1 (DoD, 2015).

The normalized IRLs associated with a particular technology need to be averaged to come up with a representative number indicating how well that particular technology relates to the other subsystems or technologies in the system. The irli accomplish this. Note that the normalization factors ‘reset’ the metric at each level of maturity, which reduces the possibility of one very mature component masking a less mature one in the metric. MRL normalizations and definitions are shown in Table 2. Note that the normalized MRL (mrl) does not replace the existing MRL, but is an intermediate step needed for SRL calculation as is the normalized IRL (irl).

Table 2. Normalized manufacturing readiness level definitions


The SRL metric is formed by multiplying the normalized MRL, the TRL, and the mean of the normalized IRL in a scalar contraction (dot product) such that each component SRLi has a value from 1 to TRL as does the scalar SRL. System readiness definitions and nomenclature are shown in Table 3.

Table 3. System readiness level definitions and nomenclature

Numerical Examples

For simplicity and clarity, this article shows three numerical-only examples using a hypothetical system with three technologies, as shown in Figure 6.

Figure 6. Example system with three component technologies

arj-78-article-1-figure-6Each technology has an associated TRL and MRL. Each Interface has an associated IRL. Notationally, this will be of the form shown in equations (6), (7), and (8).


Early Technology with Adequate Interfaces

The author assumes a simple, three-component system, with components 1, 2, and 3 funded at the 6.2—applied research level—using the following values for MRL, TRL, and IRL shown in equations (9), (10), (11), and (12).

arj-78-article-1-equation-4The first step is to calculate the normalized mrl and irl using the equations from Tables 1 and 2. Because this is 6.2 funded, the m* normalization factor is 3 from Table 2, indicating that we expect 6.2 funded technologies to be at an MRL of 3 before progressing. Likewise, the  i* normalization factor is 2 from Table 1. Normalized values are shown in equations (13), (14), and (15).

The SRL vector is calculated from the products of the normalized mrl, average normalized irl, and TRL vectors using the formulas from Table 3, shown in equations (16) and (17).


SRL = 2.33 while the average TRL is 3, indicating a slight lag in at least one interface. From the normalized MRL, one can conclude that the system is at a mostly appropriate level of manufacturing readiness with two components at an mrl of 1 and one at 0.66. SRL1 and SRL2 are at 1.5, slightly lagging behind the technology readiness of 2 and 3 due to some interface development that needs to occur. SRL3 = TRL3 = 4 indicates that this technology is at an appropriate level of interface and manufacturing readiness. The metric indicates to management that it is time to devote additional resources to the interfaces of technologies 1 and 2 before pushing ahead in further component or system development.

It is very important to conduct the early phases of interface readiness, which involve subject matter experts from different fields exchanging information and ensuring that there exists an interface solution. If this is skipped, then at the demonstration and prototyping levels of 6.4 research, many technology choices must be revisited because the technologies have matured separately and are becoming incompatible. Revisiting technology choices may then result in program delays, cost overruns, or mad scrambles to prepare for demonstrations or program cancellations. The classic case of this is thermal management, when a new technology becomes available with thermal management as an afterthought, and the legacy platform for which it is intended becomes overwhelmed with the new thermal load. The thermal issues associated with 5th generation aircraft (Majumdar & Kjelgaard, 2015) are a result of thermal interface as an afterthought. The opposite appears to be happening in the semiconductor industry in which thermal management is a very active area of research in anticipation of higher thermal loads on microchips in the near future.

Mid-level Technology with Lagging Interfaces

In a more abbreviated form than the previous example, we assume a simple, three-component system with components 1, 2, 3 funded at the 6.4 – demonstration level. MRLi = {3,5,5}; TRLi = {6,4,5}; TRL = 5, and the IRL has the following values: IRL12 = 3, IRL13 = 2, and IRL23 = 2. The SRL calculation is as follows:



The fact that SRL = 1.3, but there are TRLs at 6 and 4 and an average TRL of 5, alerts management there are serious manufacturing and interface issues, probably due to neglect in early technical development. Note that the mrl1 is 0.6 and is slightly lower than the other two; the SRLi are very nearly all at 2; and the TRLi are quite high—at 6, 4, and 5—due to the irl being much lower. This alerts management that emphasis needs to be placed on developing interfaces. Further component maturation is very risky and very likely counter-productive. It makes no sense to continue and pursue more mature technology that may or may not work in the intended environment or with the other subsystems. This system is headed toward program-killing safety, thermal, control, electrical, or other integration and deployment issues.

Advanced Technology with One Lagging Interface

The author assumes a simple, three-component system, with components 1, 2, and 3 funded at the 6.7 — operational systems development level. MRLi = {7,7,7}; TRLi = {7,7,7} and the IRL has the following values: IRL12 = 7, IRL13 = 7, and IRL23 = 4. The SRL calculations are as follows:

SRL = 6, but the TRLs are all at 7. This alerts management that there is at least one interface or manufacturing issue. Examining the component SRLs reveals that SRL1 = TRL1 = 7, but the other two SRLs lag TRL, indicating that the interfaces from component 2 to 3 are lagging and should be addressed before developing the component technologies further.

Practical Example—High Energy Laser System

Note: This is an example and not representative of any particular system. A high energy laser system is in early research and development, primarily funded by 6.2 and 6.3 sources. It is composed of at least the following subsystems: laser, beam director (BD), thermal management (TM), electrical management (EM), structural support (Struct), atmospheric propagation (Atmos), target, target acquisition, tracking, pointing (ATP), and battle management and controls (BM). A TRL assessment might be as follows (Table 4).
arj-78-article-1-table-4Note that the target TRL and MRL are “n/a”  because the system does not involve building the target, but the atmosphere and ATP form an interface with the target so an IRL is associated with the target, but no TRL.
An IRL matrix might look as shown in Table 5, if most effort had been placed into developing the laser, beam director, ATP algorithms; and target information, but not much effort placed on ‘system’ issues, such as the electrical or thermal management systems or the controls architectures. For simplicity, only the upper half of the matrix is shown assuming that IRLij = IRLji.
Table 5. Sample IRL Ratingsarj-78-article-1-table-5Not every component has every kind of interface so that the n/a values in Table 5 are simply not part of the calculation. Applying the equations in Tables 1, 2, and 3 yields the results shown in Table 6, assuming normalization by the 6.3 funding values from Table 1 and Table 2.The SRL = 2.19. The average TRL = 3.1.
Table 6. Sample IRL, MRL and SRL Components
arj-78-article-1-table-6The average SRL is below the average TRL, indicating that there are some integration or manufacturing issues that should be addressed before the components are developed further. Specifically, the laser subcomponent itself is a TRL 4, with an appropriate level of manufacturability; however, its average IRL is the lowest of any of the other subsystems. Such a system is in danger of developing a main component that cannot be integrated, demonstrated with a prototype system at an appropriate level, or that will come up with extensive integration issues late in development. These integration issues may prove to be very costly and time-consuming to fix. It would be best to develop the laser-thermal, laser-electrical, laser-battle management, and laser control interfaces before continuing to mature the laser technology itself. The side benefit would be the ability to demonstrate early prototype laser systems rather than waiting for full maturity of the final laser to conduct any demonstrations, which would be conducive to maintaining the interest in funding this technology development effort.
Verbal System Readiness Level Definitions
The proposed mathematical definition of SRL permits a verbal definition of SRLs in a way that the Sauser definition and mathematics did not. There is one caveat to these verbal definitions: they strictly hold fast at those milestones of development where SRL = TRL. It is possible to have an SRL of 3 with TRLs of 6 by ignoring interfaces and manufacturing, in which case the following definition of SRL = 3 (Table 7) would not be accurate because the SRL metric is significantly lagging the TRL metrics. This caveat also helps ensure that one cannot inappropriately claim a high level of SRL by having one mature component mask a less mature one.
Table 7. System Readiness Level Definitions
Generalized Integration Readiness Level Definitions
The author proposes the simplified critical item lists (Table 8) for the IRLs (U.S. Navy, 2009). The simplified lists allow a wider application to physical rather than IT systems, and focus on the few truly critical milestones rather than many contributing factors. See U.S. Navy (2009, p. 6) for a comparison.
Table 8. Simplified Integration Readiness Level definitions
arj-78-article-1-table-8Use of the SRL Metric
Any time the performance or behavior of a complex system is summarized by a single number, there is inevitable loss of information and the potential for false indication. SRL and IRL have a subjective component to them, as do TRL and MRL. The existence of the SRL metric will not completely compensate for organizational or programmatic pressure to advance technologies prematurely to meet budget and schedule. It will, however, foster an awareness of the cost of doing so. The SRL metric, as defined herein, is designed to be an indication that a system or component is ready for the next step in development when the system readiness is commensurate with the technology readiness. From equations (5) and (11), where SRL = TRL at the system level and SRLi = TRLi at the component level, advancement is appropriate. Since interfaces cannot be more mature than their component technologies, SRL will lag TRL at each step of development. At that point, the program will move to a higher funding, maturity, or development category; the normalization factors will change; and SRL will once again lag TRL as shown conceptually in Figure 7. This built-in safeguard will reduce the possibility of a mature subset of the system overshadowing a less mature part and giving false indications (Kujawski, 2010). A further safeguard can be implemented by limiting the values of the normalized IRLs and MRLs (mrl and irl from Tables 1 and 2) to a maximum of 1.0, further ensuring that one mature component cannot mask a less mature one. The principle that advancement to the next level of funding or acquisition should not occur until the system readiness is commensurate with the technology readiness can and should be applied at the system level (when SRL = mean[TRLi]) and at the component technology level (when SRLi = TRLi).
This article has proposed a modification to the Sauser mathematics of IRL and SRL that allows an SRL metric that gives a clear indicator of when a component technology or system is ready for further advancement and allows for standard verbal definitions of SRL. SRL and IRL need to be incorporated into the system engineering process early in development. TRL has been a valuable metric; however, its lack of emphasis on systems issues has resulted in divergent development, where some system components are developed beyond their interfaces and manufacturing, resulting in legacy decisions that impede demonstration and integration. A useful SRL metric can help to foster more balanced and cost-effective technology development.
Bender, J. (2015, March 20). Pentagon: Here are all the problems with the F-35. Business Insider. Retrieved from
Defense Acquisition University. (2016). Research, Development, Test and Evaluation (RDT&E) Budget Activities (BAs). Glossary of Defense Acquisition Acryonyms and Terms. Retrieved from
Department of Defense. (2011). Department of Defense Technology Readiness Assessment (TRA) guidance (Revision). Retrieved from
Department of Defense. (2012). Manufacturing Readiness Level (MRL) deskbook (Vol 2.21, § 2.3). Retrieved from
Department of Defense. (2015). Operation of the defense acquisition system (DoDI 5000.02). Retrieved from
Kujawski, E. (2010, October 25–28). The trouble with the System Readiness Level (SRL) index for managing the acquisition of defense systems. Paper presented at National Defense Industrial Association 13th Annual Systems Engineering Conference, San Diego, CA.
Majumdar, D., & Kjelgaard, C. (2015, March 27). F-35 needs a bigger, more powerful engine. Retrieved from
Ross, S. (2016). Application of system and integration readiness levels to Department of Defense Research and Development. Defense Acquisition Research Journal, 23(3), In Print.
Sauser, B. J., Forbes, E., Long, M., & McGrory, S. (2009, July 20–23). Defining an integration readiness level for defense acquisition. Proceedings of International Symposium of the International Council on Systems Engineering, Singapore, Japan.
Sauser, B. J., Gove, R., Forbes, E., & Ramirez-Marquez, E. (2010). Integration maturity metrics: Development of an integration readiness level. Information Knowledge Systems Management, 9, 17–46.
Sauser, B. J., Ramirez-Marquez, J. E., Devanandham, H., & Dimarzio, D. (2007). A system maturity index for the systems engineering life cycle. International Journal of Industrial and Systems Engineering, 3(6), 673.
Sauser, B. J., Ramirez-Marquez, J. E., Magnaye, R., & Tan, W. (2008a). A systems approach to expanding technology readiness within defense acquisition. International Journal of Defense Acquisition Management, 1, 39–58.
Sauser, B. J., Ramirez-Marquez, J. E., Magnaye, R. B., & Tan, W. (2008b, May 14–15). System maturity indices for decision support in the defense acquisition process. Paper presented at 5th Annual Acquisition Research Symposium of the Naval Postgraduate School, Monterey, CA.
Sauser, B. J., Verma, D., Ramirez-Marquez, J., & Gove, R. (2006, April 7–8). From TRL to SRL: The concept of systems readiness levels. Paper presented at Conference on Systems Engineering Research (No. 126), Los Angeles, CA.
U.S. Navy. (2009). Littoral combat ship mission module program system maturity assessment guide (MAG). Washington Navy Yard, DC: Author.
Standard Technology Readiness Level and Manufacturing Readiness Level Definitions
Table A-1. Standard TRL definitions
arj-78-article-1-table-a1Table A-2. Standard MRL definitions
arj-78-article-1-table-a2TABLE A-3. DoD STANDARD FUNDING CATEGORIES

To print a PDF copy of this article, click here.


arj-78-article-1-rossDr. Sean Ross has worked at the Air Force Research Laboratory, Directed Energy Directorate, since 1994. He is currently on a career-broadening assignment in the office of the Deputy Assistant Secretary of the Air Force for Science, Technology and Engineering. He is the author of “Laser Beam Quality Metrics” textbook and frequently teaches courses on the subject. Dr. Ross led the creation of the Environmental Laser Test Facility to test high-energy laser systems and components in simulated flight environments prior to flight testing. He has been involved in power, thermal, structural and other high-energy laser integration issues for over a decade. Dr. Ross holds a BS and MS in Physics from Brigham Young University and a PhD in Optical Science and Engineering from the Center for Research and Education in Optics and Lasers, College of Optics and Photonics.
(E-mail address: