Armstrong, J. Scott. Long-Range Forecasting: From Crystal Ball to Computer. New York: Wiley-Interscience, 1978. [Petersen]
Armstrong, Willis C., et al. "The Hazards of Single-Outcome Forecasting." Studies in Intelligence 28, no. 3 (Fall 1984): 57-70. In Inside CIA's Private World: Declassified Articles from the Agency's Internal Journal, 1955-1992, ed. H. Bradford Westerfield, 238-254. New Haven, CT: Yale University Press, 1995.
This article is drawn from a longer report to the DCI/DDCI from the Senior Review Panel "on the quality of intelligence judgments preceding significant historical failures over the last twenty years or so." The original report contained 12 cases studies, of which five are presented in the article -- the first Chinese nuclear test, the Soviet ALFA-class submarine, the OPEC price increase of December 1973, the 1974 Ethiopian revolution, and the Soviet intervention in Afghanistan in December 1979. The authors' critique focuses on the perils of failing to analyze and present alternative outcomes.
Ben-Israel, Isaac. "Philosophy and Methodology of Intelligence: The Logic of Estimate Process." Intelligence and National Security 4, no. 4 (Oct. 1989): 660-718.
The author argues that, for intelligence estimates, inductively deriving conclusions from known data is the wrong method. He, then, develops an alternative methodology, based on the scientific method, which he calls the "critical method." The refined critical methodology is used to analyze the intelligence failure of the Yom Kippur War (the use of hind-sight is acknowledged here), which he attributes to the use of conventional-inductivist logic.
Berkowitz, Bruce D. "Intelligence in the Organizational Context: Coordination and Error in National Estimates." Orbis 29 (Fall 1985): 571- 596.
Berlin, Don L. "Why Intelligence Estimates Won't Mislead Us Anymore." Defense Intelligence Journal 3, no. 2 (Fall 1994): 21-35.
Department of Defense Futures Intelligence Program (DoDFIP), approved by D/DIA, 18 Oct. 1993. The article's title certainly does not tell the full story.
Best, Richard A., Jr. Intelligence Estimates: How Useful to Congress? Washington, DC: Congressional Research Service, Library of Congress, 6 Jan. 2011. Available at: http://www.fas.org/sgp/crs/intel/RL33733.pdf.
"There seems to be an emerging consensus that publicly releasing NIEs, or even unclassified summaries, has limitations. Some of the nuances of classified intelligence judgments are lost and there are concerns that public release of an unclassified summary of a complicated situation does not effectively serve the legislative process. In passing the FY2010 Defense Authorization Act (P.L. 111-84), Congress chose not to require an NIE on the nuclear ambitions of certain states and non-state actors, but rather to request biennial reports (with unclassified summaries) from the DNI."
Betts, Richard K. "Strategic Intelligence Estimates: Let's Make Them Useful." Parameters 10 (Dec. 1980): 20-26. [Petersen]
Blash, Edmund Charles II. "Strategic Intelligence Analysis and National Decisionmaking: A Systems Management Approach." International Journal of Intelligence and Counterintelligence 6, no. 1 (Spring 1993): 55-68.
Brinkley, David A., and Andrew W. Hull. Estimative Intelligence: A Textbook on the History, Products, Uses, and Writing of Intelligence Estimates. Columbus, OH: Batelle, 1979.
Brown, Harold. "The Military Planner's Challenge: Reconciling Technology with Policy." Foreign Affairs 45, no. 2 (Spring 1967): 277-290.
Brown is a former Secretary of Defense. The section of the article that is of direct interest is: "Section II: The Utility of Intelligence Estimates."
Clark, Keith. "Notes on Estimating." Studies in Intelligence 11, no. 3 (Summer 1967): 55-64
The author notes that he has "drafted, chaired. or otherwise participated in many of the National Estimates." Here, he seeks "to identify, primarily from the ONE viewpoint, some recurrent dilemmas and common pitfalls in producing estimates, to note different ways of coping with these, and to suggest some main sources of strength or weakness, as well as some avoidable wastes of time and effort."
Clark, Robert M. Intelligence Analysis: Estimation and Prediction. Baltimore, MD: American Literary Press, 1996.
Cohen, Raymond. "Threat Assessment in Military Intelligence: The Case of Israel and Syria, 1985-86." Intelligence and National Security 4, no. 4 (Oct. 1989): 735-764.
"During the ten-month period September 1985-June 1986 the Israeli Army was placed on heightened alert on at least five separate occasions as a result of forebodings of a possible Syrian attack.... In response to Israel's heightened states of alert the Syrian Army itself took precautionary measures and tension rose dangerously." Why, then, did war not result? This is the interesting question with which Cohen plays provocatively.
Dulles, Allen W. "Intelligence Estimating and National Security: An Address, January 26, 1960." Department of State Bulletin 42 (14 Mar. 1960): 411-417.
Ellsworth, Robert F., and Kenneth L. Adelman. "Foolish Intelligence." Foreign Policy 36 (Fall 1979): 147-159.
Petersen: "[C]ritical of CIA's record in providing valid estimates over the years."
Feder, Stanley A. "Forecasting for Policy Making in the Post Cold-War Period." Annual Review of Political Science 5 (Jun. 2002): 111-125.
Ford, Harold P. CIA and the Vietnam Policymakers: Three Episodes, 1962-1968. Washington, DC: History Staff, Center for the Study of Intelligence, Central Intelligence Agency, 1998.
Clark comment: The former officer in CIA's Office of National Estimates and, later, Vice Chairman of the National Intelligence Council (NIC) provides "a candid view of the CIA's intelligence assessments concerning Vietnam during three episodes between 1962 and 1968 and the reactions of senior US policymakers to those assessments." (Foreword, i) The episodes presented are:
"Episode 1, 1962-1963: Distortions of Intelligence";
"Episode 2, 1963-1965: CIA Judgments on President's Johnson's Decision to 'Go Big' in Vietnam"; and
"Episode 3, 1967-1968: CIA, the Order-of-Battle Controversy, and the Tet Offensive."
Anderson, Intelligencer 9.3, calls Ford's book "one of the best studies on the Vietnam War." Goulden, Intelligencer 10.2, is similarly very positive about this book, noting that the author describes the policy debates in Washington "[w]ith consummate skill." For Shryock, IJI&C 13.4, this is an "exceptional piece of work." Three quibbles that the reviewer has with the work are that "Ford's footnotes are sometimes a mite meaty,... the index is maddeningly incomplete,... [and] there is no bibliography."
According to Bob Brewin, "Web Docs Show NSA Forecast Bloody Tet Offensive," Federal Computer Week, 2 Oct. 1998, Ford's book shows that "[i]ntercepts of enemy radio communications collected and collated" by NSA "provided U.S. commanders in Vietnam with more than two weeks' notice of the bloody 1968 Tet Offensive.... [Ford] told Federal Computer Week that he received permission from NSA to refer to its still-classified history of NSA operations in Vietnam."
Ford, Harold P.
1. Estimative Intelligence: The Purposes and Problems of National Intelligence Estimates. Washington, D.C.: Defense Intelligence College, 1989. Rev. ed. Lanham, MD: University Press of America, 1993. [pb]
A UPA advertisement identifies Ford as a former "Staff Chief of CIA's Office of National Estimates; Chief of a major CIA station overseas; a Staff Officer of the Senate Select Committee on Intelligence; a National Intelligence Officer (NIO); Vice Chairman of the National Intelligence Council (NIC); and Acting Chairman of the NIC."
Surveillant 2.6 notes that this is "[u]sed as a textbook at the Defense Intelligence College." According to Herman, I&NS 9.3, Estimative Intelligence consists of "nine brief chapters, each with easy-to-read 'bullets' of factors and conclusions.... [It] includes student exercises and a bibliography, plus case-summaries and classic papers as Annexes."
See also review by CIA's DDI, 1962-1966: Ray S. Cline, "Analyzing Past Experiences to Look into the Future," Foreign Intelligence Literary Scene 9, no. 3 (1990): 6-8.
2. Estimative Intelligence. The Intelligence Profession Series, No. 10. McLean, VA: Association of Former Intelligence Officers, 1993.
Surveillant 3.4/5: "[D]istilled from ...1989 Defense Intelligence College work."
Ford, Harold P. "The Primary Purpose of National Estimating." Studies in Intelligence 35 (Fall 1991): 69-79.
Ford, Harold P. "The US Government's Experience with Intelligence Analysis: Pluses and Minuses." Intelligence and National Security 10, no. 4 (Oct. 1995): 34-53.
This is a brief-but-informed jaunt through the history of U.S. intelligence analysis efforts, with the focus on what has come to be called "estimative intelligence." It culminates in an overview of current problem areas in U.S. analytic production, and offers in a bang-bang fashion ways of lessening the "many hazards besetting intelligence analysis." Much longer presentations of where we are and where we should be going have been written that say much less than this article.
Return to Analysis Table of Contents