To print a PDF copy of this article, click here.
In any business, trade, or profession, experience matters. Not surprisingly, the public tends to look at experience as a necessity when personal safety is paramount. Professions like the medical, transportation, and construction industries all rely heavily on experience. They take considerable time to qualify their respective corps through various experience incubators like internships, fellowships, apprentices, etc.—all on the job. They learn by “doing.” Without “doing,” these personnel may face challenges later they cannot easily overcome when “know-how” matters the most. The defense acquisition profession is no different. Experience has always been a vital constituent component. This article addresses the experience catalysts that matter most to the Defense Acquisition Workforce.
For practical reasons, many professions use quantitative measures such as “hours” or “years” to measure the experience levels of their employees. Such measures not only give these trades more confidence, but also give the public more confidence. Assured and demonstrated competencies are universally recognized as a vital necessity since inexperience could lead to life threatening consequences. Many of these same professions are backed up by licensing (or certification) boards coupled with front-line experts focused on maintaining minimum standards.
For example, after passing their medical boards, burgeoning surgeons spend years of internship practicing their craft under the watchful eye of experienced surgeons before they ever get sanctioned as qualified surgeons. Entry-level military and commercial airline pilots must achieve a minimum number of successful flight hours under variable operating conditions before they can climb into the far left seat as qualified pilots-in-command. In general, fundamentals like educational achievement, aptitude, previous job performance, etc., serve as initial career screening mechanisms. But, are there any innovative experience-producing methodologies or modalities that can appreciably accelerate experience or shrink the time it takes to achieve it?
If so, many professions including the Defense Acquisition Workforce could benefit since their certification levels rely heavily on experience. Twenty-one years after the Defense Acquisition Workforce Improvement Act (DAWIA) of 1990 became law, experience is still an essential component. If its importance becomes minimized, experience shortcomings would invariably surface and could delay the fielding of indispensable weapon systems. Now is the time, with the federal government’s current wave of retirements and impending significant budget cuts, to take a closer look at the experience variables in the acquisition workplace performance equation. Essentially, it’s time to answer the question: Acquisition experience gaps—what matters and what does not?
This investigative effort used a phenomenographic methodology (i.e., aggregate views drawn from personnel experiences) by surveying a wide range of acquisition professionals (e.g., program managers; systems engineers; logisticians; contract specialists; and budget, cost estimators, and financial managers) in various product lines (e.g., ships, tanks, aircraft, satellites, munitions, information warfare, etc.) and services (e.g., information technology, research, security, etc.), and their views on experience catalysts. Answers to these survey questions would confirm the key experience solutions that fortify the professional acquisition corps’ capabilities and combat the uncertain and sometimes turbulent and impending programmatic challenges.
The survey separated experience catalysts (EC) into three tiers: Foundational (Tier 1), Enhancers (Tier 2), and Accelerators (Tier 3). Isolating ECs in this way, the surveyors believed, might give way to a more definitive analysis later. Ultimately, this partition could also help explain experience gateways and validate the prevailing obstacles (real or artificial) that could be interfering (in the form of barriers) with experience gains along the acquisition “experience building” pathway. The total sum of these factors would look something like the equation shown here:
EC = ∑ (Tier 1i + Tier 2i + Tier 3i) – Barriersi
A total of 1,414 Defense Acquisition Workforce personnel (1,236 government, 152 military, and 26 support contractors) responded to this survey. The results reinforced both the importance and influence of a wide range of experience catalysts operating inside and outside the workplace. However, the data exposed a few that were not operating at expected levels and also generated several “Aha!” moments.
1st Tier: Experience Foundational
Inarguably, many professions rely on enduring academic foundations. Depending on the specific functional area(s) a member of the Defense Acquisition Workforce chooses to pursue, these academic foundations also serve as formal learning toll gates before personnel arrive on the job. Of course, well-described job competencies reinforced by definitive performance expectations ensure that personnel are properly placed and appropriately guided. Systems engineers should be ready to apply engineering basics; contract specialists should be ready to carefully evaluate written agreements; and cost estimators should be steeped in math sufficient to comfortably work with budget and cost estimate equations.
Despite the profession, however, these formal foundational learning gates are less than half of the total learning equation. The remainder actually occurs at the workplace. In fact, more than 70 percent of most new knowledge and skills actually take place at work through a combination of informal and incidental learning (Good & Brophy, 1990). This is where the workforce tests their inherent capabilities every day. Where do these foundational experience catalysts play in all of this? They appear to take root more in the context of these informal and incidental learning methods (i.e., “learning by doing”). If so, what did the Defense Acquisition Workforce actually say about the effectiveness and value of these foundational experience catalysts early on while actually working “in” the job? What mattered most?
The survey respondents rated the importance of a broad range of experience factors. As Figure 1 indicates, the results were consistent with previous research. On-the-job training mattered the most. Knowledge sharing with colleagues and challenging work trailed very closely behind. Several respondents expressed that “learning and understanding others’ experiences reinforced their own.” Not surprisingly, knowledge sharing can have far-reaching considerations since knowledge is seen as “the most strategically important resource which organizations possess and a principal source of value creation” (Cummings, 2003).
Figure 1. T1: Foundational
Supervisory guidance represented the next data point. One respondent echoed the views many others shared. She claimed that “having a well-trained supervisor who is a great teacher, allowing me to fly semi-alone … built [my] confidence, knowledge, and courage to complete more challenging tasks.” The next lower grouping included DAWIA classroom training, formal mentorship, professional development, well-defined organizational processes, online training, and certification standards. Unexpectedly, three of these seven data points (DAWIA training, well-defined organizational processes, and certification standards) all scored noticeably low and could be explained for several reasons.
Probable reasons why survey respondents gave DAWIA classroom training a low score:
- DAWIA classroom training’s value could be muted compared to other more dominant experience catalysts. Some respondents expressed that classroom experience will “never be able to replace OJT [on-the-job training], mentoring, or knowledge sharing at work.” Others emphasized that DAWIA classroom training is “rather generic and does not actually teach enough of the job specifics.”
- Students might be showing up too early (or late) for training during their career. Several respondents noted the difficulty in keeping up with additional training demands.
- Students forgot what they learned before they could apply it.
- DAWIA classroom training could possibly have a looser connection to experience in its current form.
- The benefits of DAWIA classroom training might not be well-understood, especially the connection to performance outcomes.
In a recent report, the U.S. Government Accountability Office (GAO) declared that without appropriate outcome metrics, acquisition, technology, and logistics programs will be “unable to demonstrate how certification training actually contributes to organizational performance results” (GAO, 2010). What the GAO underscored is tough to demonstrate without a comprehensive program that tracks behavioral changes at work. The discovery that as much as 90 percent of training resources spent on the design, development, and delivery of training events yield application results of only 15 percent (Brinkerhoff, 2006) makes training an easy target for scrutiny.
In the context of Donald Kirkpatrick’s well-known Four Levels of Learning Evaluation, the first two learning levels (Reaction , and Learning ), have been relatively easy to demonstrate during the classroom delivery timeframe. Level 3 (Behavior) and Level 4 (Results) have been a lot tougher to confirm. Some researchers assert that if Level 3 evaluations were conducted as part of existing career development and performance reviews, then it might “improve, explain, control, and predict performance although managers must be willing to observe, document, and evaluate the desired behaviors” (Mayberry, 2005). Even “modest supervisor involvement before and after the training can have a significant impact on whether trainees use their newly developed skills” (Bassi and Russ-Eft, 1997). Other studies have shown that “the more managers are trained in how to support and coach the skills their employees learn, the more those skills will be used and sustained in the workplace” (Leimbach & Maringka, 2009).
Decades ago, the DoD instituted a formal performance evaluation program for all its employees to signal the importance of training. In 1958, legislators more than likely expected that the Government Employees Training Act would improve performance and prepare personnel for future advancement (Government Employees, 1958). In 1962, the subsequent Federal Salary Reform Act required an acceptable level of competence determination for granting General Schedule within-grade increases; provided for the denial of the within-grade increase when performance is below the acceptable level; and authorized an additional step increase for high-quality performance (Federal Salary, 1962). While these formal evaluation measures have continued to evolve, they have not, however, specifically traced personnel performance to training activities. Educators have generally assumed that training focuses on the required knowledge, skills, and abilities (KSA) necessary to perform and improve assigned duties within the workplace. In fact, plenty of literature substantiates this probabilistic connection. However, many other intervening factors complicate the relationship including individual attitude, motivation, cultural realities, learning self-efficacy, age, etc. (Bassi and Russ-Eft, 1997). Making a deterministic forecast is difficult.
Other factors including team structures, incentives, use of analytic tools for capturing and analyzing information, and psychological safety tend to moderate the association between experience and performance improvement (Edmondson, 1999). Nonetheless, the private business sector has found training to have a positive impact on profitability (Cosh & Hughes, 2003, pp. 88–95.). Many years ago, the DoD made a similar association for its Defense Acquisition Workforce and invested heavily in training.
As far as experience foundational catalysts go, several others require further introspection.
Probable reasons why survey respondents gave well-defined organizational processes a low score.
- Organizational processes may already be culturally embedded and not viewed as a distinctive element.
- Organizational processes may not represent much value and are not enforced.
Probable reasons why survey respondents gave certification standards a low score.
- The Certification Standards could be generally misconstrued.
- The Certification Standards did not go far enough or were too watered down to be significant.
- The Certification Standards’ connection to job performance was not readily apparent.
Probable reasons why survey respondents gave Communities of Practice (CoP)—another form of knowledge sharing—a low score.
- The CoP website is not a rich source of useful knowledge.
- The information posted on the CoP website is not current.
- The existence of a CoP website is not well known.
- Information on the CoP website may not be appropriately curated (e.g., information has not been properly maintained or trusted for use).
- The CoP website could represent a loss of social interaction that generally creates more value.
2nd Tier: Experience Enhancers
The impact of experience catalysts expressed as experience enhancers (T2) seemed relatively consistent to those described as foundational (Figure 2) and were very closely correlated. On-the-job training didn’t diminish in importance; neither did knowledge sharing, challenging work, or supervisory guidance. In relative terms, they all rose slightly.
Figure 2. T1 (Foundational), T2 (Enhancers)
Both classroom training and online training rose more noticeably in relative terms. The uncharacteristic rise in online training could be attributed to: (a) how online training complements certain experience foundations; or (b) the presence of more effective delivery methods (e.g., greater interactive modalities and less of a “page turner”). Traditionally, DAWIA classroom training that uses Scenario-Based Learning (SBL) methods enjoys more of an advantage than other classroom methodologies for students with relevant job experience (Clark, 2009, pp. 84–85). It gives students a chance to practice representative training scenarios alongside their peers, and reflect about their jobs while they are away from their jobs. Reflection and practice have been found to have a significant impact on experiential learning of this kind.
David Kolb, an American educational theorist, reported that in order to gain genuine knowledge from an experience, the learner “must be able to reflect on the experience as well as be willing to get actively involved in the experience; possess and use analytical skills to conceptualize the experience; and possess decision-making and problem-solving skills in order to use the new ideas gained from the experience” (Kolb, 1984). Classroom training that employs this type of SBL does just that and today is used extensively since it adheres to a performance improvement imperative rather than just the acquisition of knowledge and skills. SBL also promotes defining moments by exposing an individual’s strengths and weaknesses. By imitating something real, SBL has been shown to pay unmistakable experience dividends by igniting the senses. SBL has already found its way into organizations that vitally depend on training. Soaked with real-world conditions, SBL tests an individual’s ability to demonstrate how certain critical competencies prevail (or not).
Captain Chesley “Sully” Sullenberger III, a former U.S. Airways seasoned pilot, experienced its value first-hand. He spent the better part of two full days every six months at the controls of an Airbus 319 flight SBL simulator while several lifetimes’ worth of disasters broke loose around him (Budiansky, 2009). At what point was he prepared for a water landing on the Hudson River when he piloted Flight No. 1549 on January 15, 2009? How many years did it take for him to turn a potential disaster into a miracle? He met his flying experience markers (in years), but up to the moment before he set his aircraft on the Hudson, an SBL simulator allowed him to fly at the edge of the flight envelope and test him for just about any contingency—except a water landing. The Airbus 319 isn’t a watercraft, but Sully knew he had to treat it like one given the threatening outcome of two failed engines. His many years as an experienced “line” pilot combined with recurring scenario-based simulator training helped him tackle “the unexpected” and ultimately save 155 lives that day.
Aside from their longstanding presence in the flying community, simulators also show promise for many other professions that require continuous practice and steady reinforcement. Virtual simulators were previously an expensive proposition. Not anymore. Now, high-fidelity virtual simulations and the introduction of gaming using 3-D capability are relatively inexpensive and widespread. They could eventually become commonplace in many workplace settings. When that occurs, they might have an even greater impact on experience gains for many professions where workers can safely practice a wide range of challenges preloaded with uncertainty, but customized to their respective on-the-job training settings.
3rd Tier: Experience Accelerators
The data associated with this last tier resulted in several interesting observations. First, fewer correlations were noted with 1st and 2nd tier factors. Second, professional development, well-defined organizational processes, and formal mentorship took a marked leap in importance as accelerators (Figure 3). Third, challenging work and certification standards took visible dips. What caused certain experience catalysts to rise in importance and others to fall?
Figure 3. T1 (Foundational), T2 (Enhancers), T3 (Accelerators)
Probable reasons to explain why the T3 experience factors rose in importance.
- Professional development. This factor rated off-the-job training and the potential knowledge gains found outside the workplace on supplementary/complementary subjects and/or interactive knowledge sharing venues with leaders in their same fields.
- Professional development opportunities. This factor rated the importance of interacting and knowledge sharing with colleagues outside the workplace.
- Well-defined organizational processes. This factor rated tangible benefits of more definitive written organizational guidance that might have been less obvious before. Research has shown that learning from direct experiences depends critically on organizational processes that generate experiences (Schultz, 2001).
- Formal mentorship. This factor rated the importance of personnel seeking advice and counsel from more seasoned professionals in their same career fields in their own work environment. One respondent commented that “having a hands-on mentor made a world of difference.” Another stated that “having a hands-on mentor at the start of their career would have made a world of difference.”
Probable reasons to help explain why some T3 experience factors dropped in importance.
- The dip in challenging work could be attributed to three probable causes:
- The work at hand may no longer be challenging enough and could be holding people back.
- Work overload—good work is rewarded with more work without the time to adequately learn it.
- A complicating effect of increased administrative burden (seen by some as busy work) is too much sidebar work to promote any real preferential experience gains.
- The dip in certification standards (and the lowest of all experience accelerators) could also be attributed to three probable causes:
- The certification standards contain poorly described benefits—professional and personal payback are not readily apparent.
- Achievement thresholds are too low or less relevant to current jobs.
- Certification levels were awarded too long ago and are less relevant today.
To understand their views of experience barriers, the survey respondents were asked to comment on the lack of or reduction in certain experience catalysts. As Figure 4 shows, the barriers followed a close inverse correlation to experience accelerators. These barriers did not necessarily predominate, but they did seem to induce a certain experience drag.
Figure 4. T1 (Foundational), T2 (Enhancers), T3 (Accelerators), Barriers
The lack of well-defined organizational processes (also seen as an experience accelerator
- Outdated processes no longer applied.
- Support was reduced for existing organizational processes.
- Ambiguity surrounded the issue of whether certain key organizational processes even existed.
- Guidance was poorly conveyed, without adequate explanation or appropriate justification. One respondent stated that the lack of published work processes curbed his experience gains.
The lack of formal mentorship (also seen as an experience accelerator when visibly present) emerged as a barrier, suggesting that some personnel require more coaching.
The lack of participation in CoPs was neither a barrier nor considered a substantial experience factor in any one of the three tiers. While CoPs can give access to a tremendous set of colleagues steeped in relevant knowledge and experience, they appear to have less of an impact on experience growth than expected.
Certification standards were not seen as a barrier, suggesting that the workforce did not necessarily view them as inhibiting experience gains or helping to achieve them.
The data in this study confirmed the substantial influence of certain experience catalysts where they tend to predominate—in the workplace. Understanding the correlation and value of these high flyers can have a marked impact on individual performance and acquisition outcomes if fully exploited. The experience catalysts, operating in a less influential state, could have a noticeable impact as well. If appropriately recognized (and in some cases either clarified or re-energized), they could serve as a powerful force multiplier for even more experience gains.
Members of the Defense Acquisition Workforce participating in this study reaffirmed the major experience gains achieved by work-related experience catalysts. Ideally, the sooner that formal training and informal training converge, the greater will be the impact of off-the-job training that better prepares the workforce for many more uncertainties in the workplace (Figure 5).
Figure 5. Reducing the Gap Between Off-the-Job and On-the-Job Catalysts
Convincing organizations to embrace themselves as informal learning organizations where the preponderance of experience actually takes root (i.e., on-the-job) could serve as a crucible for many experience catalysts. Toward that end, the following recommendations are warranted for defense acquisition operating units:
Codify yourselves as Learning Organizations
Recognize the wide range of experience catalysts found in the workplace and how they can favorably impact organizational outcomes. Institute and monitor with regular frequency the effect of these experience catalysts inside the organization. Adjust as required. Reduce the barriers that might be limiting certain experience gains. More specifically:
- Keep the work challenging and in perspective: The Defense Acquisition Workforce expects to be challenged—a key part of their professional growth. Workers tend to stay at the job and remain focused when the work is challenging and relevant; they leave when the work is not.
- Capitalize and promote knowledge sharing opportunities. Build a flexible and enduring information architecture warehouse that contains actionable information that personnel can tap freely. Provide easy access to sources of expertise. It deepens their knowledge base, expands perspectives, and fuels their experience engine. Without open and continuous dialogue, competency gaps are more likely to occur, and experience growth might plateau and limit organizational gains. Promote knowledge sharing media like social networks. Personnel also need slack time and decision-making autonomy to benefit from access to new knowledge, regardless of the source (Haas, 2006). Reward personnel for integrating and applying new knowledge when it creates organizational performance gains.
- Get supervisors involved in the training process before and after the event. With greater involvement, training can have more relevance and create more favorable impacts back on the job. The most important environmental factors at work affecting training transfer include “discussions with the supervisor on the use of new learning, the supervisor’s involvement or familiarity with the training, and positive feedback from the supervisor” (Nijman, Nijhof, Wognum, & Veldkamp, 2006). Supervisor commitment is crucial in validating the usefulness of training.
- Clearly articulate and punctuate the effectiveness of organizational processes. Keep processes current, effective, and relevant. Communicate their usefulness with regular frequency. Revise or terminate processes that have outlived their usefulness. Do not change what is working well for the sake of change.
- Promote and support professional development opportunities. Broaden employees’ knowledge by giving them an opportunity to reap the experiences and effective practices of others. Encourage professional relationships and future experience networks that employees can leverage for years to come. Make an organization stronger by combating competency gaps, thereby helping to break down outdated mental models.
- Promote mentorship. Draw from the rich experiences of seasoned, introspective, and proven leaders. They can help build a sustainable career pathway for personnel who are looking to widen their experience gains as they pursue their professional careers.
- Recognize the efficacy of DAWIA training. Ensure employees are ready for the training and the training is meeting their needs. Provide useful and timely feedback to the training communities.
- Recognize the value of on-the-job activities. Explore immersion days and offsites to promote experience gains for personnel back on the job, and target individual and organizational performance.
The following recommendations are warranted for defense acquisition training organizations:
- Continue to tighten the connection between off-the-job training and on-the-job training. Learners need to understand the connection by witnessing the connection. The clearer the link between the skills taught and the skills required at work, the more newly acquired skills will stick. Make it truly experiential. Validate the learning objectives taught in class with outcomes in the field through measurable follow-up initiatives later at work. Specialize the training by mimicking learners’ work environments through methods that ignite the senses. View training courses as training workshops. “The road to exceptional performance is the result of deliberate practice” (Colvin, 2010).
- Maximize SBL. Few training techniques emulate actual work environments better. SBL tests the workforce under realistic conditions and gives them a chance to show their grit without the threat of dangerous consequences. It also brings together both cognitive (e.g., mental processes, knowledge application, etc.) and affective (e.g., feelings, attitude, etc.) behaviors, thereby increasing the quality of the experience. “Everything depends on the quality of the training experience” (Dewey, 1998).
- Reinforce the benefits of certification standards. While it should have bearing on upward mobility, it should not be the principal motivation. Many respondents viewed getting their certifications as a way to get promoted and sought training accordingly.
- Monitor the usefulness of knowledge sharing media like CoPs and others, especially social media. Either re-invigorate certain CoPs that have dropped sharply in popularity or replace them with more promising knowledge sharing methods. If seen as invaluable, personnel will use them. CoPs can provide the Defense Acquisition Workforce tremendous access to a wider experience network, but such experience has to go beyond simple data transmission. Research evidence shows that knowledge sharing methodologies involving personal interactions are superior to those involving only document exchanges alone. “Knowledge often needs to be carefully adapted to a new context in order for it to be effectively utilized” (Leonard-Barton, 1988).
A follow-on study that tracks specific behavioral changes associated with the experience catalysts discussed in this article would help describe the weighting and progression of these experience catalysts.
In today’s budget-tightening environment amid increased public scrutiny of every dollar the DoD spends, the Defense Acquisition Workforce is facing growing pressure to make every dollar for its goods and services count. While experience has and will continue to be a fundamental component of the human capital development equation, it is vitally important that the DoD recognize what experience catalysts matter the most to the Defense Acquisition Workforce. Twenty years from now, experience inside the Defense Acquisition Workforce will matter just as much as it did when Congress voted the DAWIA into law over 20 years ago. The only difference might be that the seam between off-the-job training and on-the-job training will disappear. When the Defense Acquisition Workforce is tested through intellectual workouts that mimic their on-the-job conditions, performance outcomes will likely rise. Exercises like Air Force Space Command’s Guardian Challenge that now includes its acquisition arm are helping achieve that goal (Tremaine, 2010).
The Defense Acquisition Workforce would be well-served if it recognizes the importance of experience catalysts—even the ones operating in the lower bands. Granted, many variables are involved in the experience equation. However, to maximize the equation the workforce must:
- Continuously practice their craft at work in what has long been serving as on-the-job laboratories.
- Apply their intellectual mettle in the face of challenging work with supervisors and mentors close by.
- Consistently share relevant information through a highly collaborative environment in a wide range of media.
- Recognize the connection between training and certification.
- Continuously think beyond yesterday’s beliefs without getting trapped by competency gaps that could prevent experimentation with more suitable and effective alternatives. Past experience can sometimes create blind spots and interfere with the need for innovation or modernization—something the Defense Acquisition Workforce or any other profession can ill afford. KSAs are so tightly connected to experience that they could become too grounded in yesterday’s beliefs. In other words, the same attributes that once yielded conventional wisdom can sometimes produce fixed mindsets, superstitious learning (e.g., single perspectives, learning the wrong things, etc.), or competency traps and erroneous inferences (Levitt & March, 1988).
Before 1947, engineers believed the speed of sound represented a physical barrier for aircraft (and pilots) because the formation of a violent shock wave would induce catastrophic aerodynamic effects and cause complete flight control failure. Those beliefs changed when Chuck Yeager broke the sound barrier in the Bell X-1 Glamorous Glennis on October 14, 1947. Similarly, other technical beliefs had to change well before Neil Armstrong could walk on the moon on July 20, 1969.
Implementing these actions would fully energize the confederation of experience catalysts and noticeably influence performance gains.
As Oscar Wilde said over a hundred years ago, “Experience is the name every one gives to their mistakes” (Wilde, 1892). Consequently, the Defense Acquisition Workforce needs the time to practice and learn from their mistakes just like any other profession, and can ill afford any experience shortfall that results in weapon systems delays for warfighters serving in harm’s way. Warfighters depend on the Defense Acquisition Workforce to get it right the first time—and that’s the only “Aha!” that really matters.
To print a PDF copy of this article, click here.
Col Robert L. Tremaine, USAF (Ret.) is the associate dean for Outreach and Mission Assistance at the Defense Acquisition University West Region. He is a retired Air Force colonel with 26 years’ experience in air, missile, and space weapon systems acquisition. He holds a BS from the U.S. Air Force Academy and an MS from the Air Force Institute of Technology. He is level III certified in both the program management and systems planning, research, development, and engineering career fields. He is also a graduate of the Canadian Land Force Command and Staff College in Toronto, Ontario, Canada; and the U.S. Army War College in Carlisle Barracks, PA.
Bassi, C., & Russ-Eft, L. J, (1997). Do it and understand! The bottom line on corporate experiential learning, what works: Assessment development and measurement. Alexandria, VA: American Society for Training and Development.
Brinkerhoff, R. O. (2006). Telling training’s story: Evaluation made simple, credible, and effective. San Francisco: Berrett-Koehler Publishers.
Budiansky, S. (2009, May 14). How Capt Sully knew what to do. Men’s Journal. Retrieved from http://www.mensjournal.com/flight-simulators
Clark, R. (2009). Accelerating expertise with scenario-based learning. T+D Magazine. Retrieved from http://www.clarktraining.com/content/articles/ScenarioBasedLearning.pdf
Colvin, G. (2010). Talent is overrated: What really separates world-class performers from everybody else. New York, NY: Penguin Group (USA).
Cosh, A., & Hughes, A. (2003). The relationship between training and business performance (Research Report No. RR454). ESRC Centre for Business Research – University of Cambridge. Retrieved from http://www.cbr.cam.ac.uk/pdf/RR454.pdf
Cummings, J. (2003). Knowledge sharing: A review of the literature. The World Bank. Retrieved from http://lnweb90.worldbank.org/OED/oeddoclib.nsf/DocUNIDViewForJavaSearch/D9E389E7414BE9DE85256DC600572CA0/$file/knowledge_eval_literature_review.pdf
Dewey, J. (1998). Experience & education: The 60th anniversary edition. Urbana-Champaign, IN: Kappa Delta Pi, University of Illinois.
Edmondson, A. (1999). Psychological Safety and Learning Behavior in Work Teams. Administrative Science Quarterly, 44, 350-383.
Federal Salary Reform Act of 1962, Pub. L. 87-793, pt II §501 et seq. (1962).
Good, T., & Brophy, J. (1990). Educational psychology: A realistic approach. New York: Holt, Rinehart, & Winston.
Government Employees Training Act of 1958, 5 U.S.C., chap 41 (1958).
Haas, M. R. (2006). Knowledge gathering, team capabilities, and project performance in challenging work environments. ILR Collection, Cornell University ILR School. Retrieved from http://www.noaminfo.com/myblog/wp-content/uploads/knowledge-gathering.pdf
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. New Jersey: Prentice Hall.
Leimbach, M., & Maringka, J. (2009). Learning transfer model: A research-driven approach to enhancing learning effectiveness. Retrieved from http://wilsonlearning.com/images/uploads/pdf/Learning_Transfer_Approach.pdf
Leonard-Barton, D. (1988). Implementation as mutual adaptation of technology and organization. Research Policy, 17, 251-267.
Levitt, B., & March, J. G. (1988). Organizational learning. Annual Review of Sociology, 14, 319–340.
Mayberry, E. (2005). Kirkpatrick’s level 3: Improving the evaluation of e-learning. Retrieved from http://www.astd.org/LC/2005/0505_mayberry.htm
Nijman, D. J. J. M., Nijhof, W. J., Wognum, A. A. M., & Veldkamp, B. P. (2006). Exploring differential effects of supervisor support on transfer of training. Journal of European Industrial Training, 30, 529–549. Retrieved from http://www.emeraldinsight.com/journals.htm?articleid=1571382&show=html
Schulz, M. (2000). Organizational learning. Retrieved from http://www.unc.edu/~healdric/Classes/Soci245/Schulz.pdf
Tremaine, R. L. (2010, September-October). Taking acquisition training to the next level: The space acquisition community tackles the Guardian Challenge. Defense AT&L, 39(5), 35–39.
U.S. Government Accountability Office. (2010). Acquisition workforce: DoD’s training program demonstrates many attributes of effectiveness, but improvement is needed (Report No. GAO 11-22). Washington DC: Government Printing Office.
Wilde, O. (1892). Lady Windermere’s fan, Act III. Retrieved from http://www.quotationspage.com/quotes/Oscar_Wilde