Critiques of Intelligence Analysis

N - Z

Nolan, Phil. "A Curator Approach to Intelligence Analysis." International Journal of Intelligence and Counterintelligence 25, no. 4 (Winter 2012-2013): 786-794.

The author argues that the U.S. "Intelligence Community (IC) is drowning in information." It "needs to move beyond an approach that uses open source information (more data) as an input to an approach that embraces open source analysis (more opinions) whatever its origin."

Pincus, Walter. "Study Faults Bush's Emphasis On Daily Intelligence Brief." Washington Post, 15 Sep. 2009, A17. []

A study by the Brookings Institution, "The U.S. Intelligence Community and Foreign Policy: Getting Analysis Right," released on 15 September 2009, concludes that "[u]nder President George W. Bush, the President's Daily Brief ... rose to 'an unprecedented level of importance,' with negative consequences for the intelligence community." The study suggested that "focusing on producing PDB items that would draw favorable comment from Bush could have skewed 'topic selection and treatment in the analytic community.'"

Rieber, Steven, and Neil Thomason. "Toward Improving Intelligence Analysis: Creation of a National Institute for Analytic Methods." Studies in Intelligence 49, no. 4 (2005): 71-77.

The authors argue that "systematic research" is needed to improve analytic practices. They would stimulate such research through establishment of a National Institute for Analytic Methods along the lines of the National Institute of Health.

Robarge, David S. "Getting It Right: CIA Analysis of the 1967 Arab-Israeli War " Studies in Intelligence 49, no. 1 (2005): 1-7.

Sometimes the intelligence process works "almost perfectly. On those occasions, most of the right information was collected in a timely fashion, analyzed with appropriate methodologies, and punctually disseminated in finished form to policymakers who were willing to read and heed it. Throughout those situations, the intelligence bureaucracies were responsive and cooperative," and the DCI "had access and influence downtown. One such example that can be publicly acknowledged" is the Six-Day War in 1967.

Russell, Kevin. "The Subjectivity of Intelligence Analysis and Implications for the U.S. National Security Strategy." SAIS Review 24, no. 1 (Winter-Spring 2004): 147-163.

Ryan, Maria. "Filling in the 'Unknowns': Hypothesis-Based Intelligence and the Rumsfeld Commission." Intelligence and National Security 21, no. 2 (Apr. 2006): 286-315.

"The now discredited intelligence on Iraq was not a 'failure' or 'mistake', but a method tried and tested by the right, of challenging the CIA on political grounds." The 1976 "Team B" exercise used the methodology, as did the 1998 Rumsfeld Commission on the ballistic missile threat.

Seliktar, Ofira. Politics, Paradigms, and Intelligence Failure: Why So Few Predicted the Collapse of the Soviet Union. Armonk, NY: M.E. Sharpe, 2004.

Peake, Studies 51.1 (Mar. 2007), points out that the author "quotes other works on the subject of the USSR's collapse, but she ignores views that do not coincide with her own," such as those that show the Intelligence Community really did not miss the Soviet collapse. Nonetheless, Seliktar has presented "an innovative approach for students and analysts alike to solve or at least study a perennial problem."

Shelton, Christina. "The Roots of Analytic Failures in the U.S. Intelligence Community." International Journal of Intelligence and Counterintelligence 24, no 4 (Winter 2011-2012): 637-655.

The author's DIA background clearly shows in where she points her finger in describing analytic "failures." Nonetheless, as she notes, "Intelligence is a battleground where insight into reality can be -- and has been -- corrupted by subjectivism. It is an arena where intellect, loyalty, and morality are put to the test."

Spivey, Robin V. "The Devil Is in the Details: The Legal Profession as a Model for Authentic Dissent." International Journal of Intelligence and Counterintelligence 22, no. 4 (Winter 2009): 632-651.

"Expanding the Devil's Advocacy concept and improving its methodology by implementing some of the best practices from the legal world could help the Intelligence Community (IC) prevent intelligence failure and improve the quality of its analytic products."

Tidwell, William. "Horrible Thought." Studies in Intelligence 2, no. 1 (Winter 1958): 65-70.

The analytic "community seems to have learned how to produce very good answers to intelligence problems without generating an undue amount of internal friction." However, "[t]he bold analysis, the sharp intuition, the long step forward, and the provocative ideas ... are almost never found in the formal papers put forward ... for the sober guidance of our planners and policymakers." There is a need to "extend our analysis in time and depth beyond present dimensions."

Treverton, Gregory F., and C. Bryan Gabbard. Assessing the Tradecraft of Intelligence Analysis. Santa Monica, CA: Intelligence Policy Center, National Security Research Division, RAND, 2008. []

This study argues that in the Intelligence Community "every agency has a separate set of research priorities and product lines.... [N]one of the agencies knows much of what its colleagues do, still less works with them consistently in testing and validating analytic techniques or in training analysts.... [W]e concluded that the establishment of a research agenda and a training and education curriculum with a Community-wide perspective is critical to future analytic tradecraft.... Our research also identified shortfalls in analytic capabilities, methodologies, and skills, and it recommends actions to take to address these gaps as well as a strategy for meeting future challenges."

U.S. Congress. House. Permanent Select Committee on Intelligence. Subcommittee on Evaluation. Iran: Evaluation of U.S. Intelligence Performance Prior to November 1978. 96th Cong., 1st sess., 1979. Committee print. Washington, DC: GPO, 1979.

This report criticizes the NIE process generally, with specific focus on how that process dealt with the collapse of the Shah's regime.

Ward, Steven R. "Evolution Beats Revolution in Analysis," Studies in Intelligence 46, no. 3 (2002): 29-36.

The author offers "counterpoint" to Carmen A. Medina, "What to Do When Traditional Models Fail: The Coming Revolution in Intelligence Analysis," Studies in Intelligence 46, no. 3 (2002): 23-28. Medina argues: "The DI's tradecraft model was developed during the 1960s and 1970s and optimized against the characteristics of that period." Today, intelligence analysts must be prepared to operate in "an era of information abundance, wellconnected policymakers, and non-traditional issues." The focus needs to be "on customer requirements, collaborative work, and less formal products."

Ward, on the other hand, suggests that Medina's critique "fails to take into account the wide variety of consumers that the DI serves, ranging from the Executive Branch and the Congress to the military and foreign intelligence partners.... [M]ore proof needs to be shown that the traditional model has failed and that significant change, much less a revolution, in the DI is needed."

Wasserman, Benno. "The Failure of Intelligence Predictions." Political Studies 8, no. 2 (Jun. 1960): 156-169.

Contrasts "intelligence" and "knowledge" in terms of what is needed for foreign policymaking.

Weiss, Charles. "Communicating Uncertainty in Intelligence and Other Professions." International Journal of Intelligence and Counterintelligence 21, no 1 (Spring 2008): 57-85.

"Reluctance to address uncertainty directly and precisely is deeply rooted in the Intelligence Community, as it is in many expert communities.... Whether the intense scrutiny that has followed the debacle over the failure to find weapons of mass destruction in Iraq will force the IC to ... adopt a more explicit approach to explaining the uncertainty that inevitably accompanies its interpretations of data and its predictions[] remains to be seen."

Young, Jay T. "US Intelligence Assessment in a Changing World: The Need for Reform." Intelligence and National Security 8, no. 2 (Apr. 1993): 125-139.

Young is a "former senior military analyst with the CIA" working on a doctorate at Ohio State University. He offers the standard criticisms expected from a disillusioned former DI hand. He does suggest centralizing assessment in CIA and abolishing DIA and INR. He would create a "National Intelligence University," to improve analyst training.

Return to Analysis Table of Contents