Notes on Analytic
Tradecraft and Counterintelligence
This is the tenth in a series of Product Evaluation Staff notes to clarify the standards used for evaluating DI assessments and to provide tradecraft tips for putting the standards into practice.
This tradecraft note addresses the aspect of counterintelligence (CI) on which the DI is most directly responsible for improved Agency performance: Countering deception operations by foreign governments and organizations aimed at distorting the conclusions and judgments of analysts' assessments.
A subsequent note will address related CI aspects on which DI substantive expertise and analytic skill can make important contributions, including (1) countering espionage by helping to identify the US secrets foreign intelligence services are most interested in obtaining and (2) analytic support of efforts to manipulate foreign intelligence operations to US advantage.
Deception is here defined as all efforts by a foreign intelligence service or adversarial group to distort and otherwise manipulate the perceptions of analysts in order to place analysts and their policy clients at a disadvantage in understanding and dealing with the perpetrating country or group. Deception operations can be divided into two closely related subsets:
For DI analysts, the first step in improving CI performance is to show increased respect for the deceiver's ability to manipulate perceptions and judgments by compromising collection systems and planting disinformation. Next, analysts must adjust the balance between speed and care in producing DI assessments. There is no "free lunch" here. The analysts' major defense for countering deception exercising increased care in evaluating information and in testing working assumptions places pressures on the DI commitment to timeliness in servicing consumer demand for tailored products.
The key to holding down the opportunity costs of countering deception is to tie the effort as closely as possible to the normal processes analysts use to expand expertise and to ensure quality and policy utility in their memorandums and briefings:
Through an investment in understanding the warning signs that a deception operation may be under way, analysts become more expert about their subjects as well as about when and how to apply their main weapons for protecting the integrity of DI assessments:
The first set of warning signs addresses the likelihood that a country or organization is engaged in an attempt to distort the analysts' perceptions:
1. Means. The country or entity (for example, terrorist group) being assessed has the experience and means to undertake sophisticated deception operations. Nearly all countries and entities given priority attention under PDD 35 have well-practiced means to deceive, often in third countries as well as at home.
2. Opportunity. The foreign country or entity is known to have countered the collection systems or platforms on which the DI analyst is particularly dependent. For example, when a target country has knowledge of the periodicity and acuity of technical collection vehicles that pass over an area it wishes to protect, analysts have to be aware that the resultant information may be incomplete if not also deliberately distorted.
Enhanced knowledge of both the reach and the vulnerabilities of collection systems will also help analysts in dealing with the everyday challenges of evaluating incomplete and ambiguous information.
3. Motive. A motive to deceive is believed to be present. Accomplished intelligence services - for example, in Russia, China, or Cuba - have the ability to mount elaborate denial and disinformation operations virtually across the board. Would they see a high payoff from distorting analyst and policymaker perceptions about the issue at hand?
The second set of warning signs focuses on anomalies in the information available to the analyst. Investment in addressing these warning signs reinforces core skills regarding what the analysts know, how they know it, and what they do not know that could affect the outcomes in ways important to policy officials, warfighters, and law enforcement officials. See, for example, Tradecraft Note No. 5, Facts and Sourcing (July 1995).
These warning signs include:
4. Suspicious gaps in collection. For example, when information central to the analysts' conclusions and judgments received through one collection channel is not supported to the extent expected from the take of other collection systems. In other words, the analysts are not receiving the range and volume of information they would expect if there was no deliberate tampering with sources and collection platforms.
5. Contradictions to carefully researched patterns. Does new information undercut the trends and relationships honed through past research? While analysts must be open-minded about the possibility of unexpected change, they should examine critically information that signals an inexplicable change, for example, in an adversary's priorities and practices.
6. Suspicious confirmations. For example, when a new stream of information from clandestine sources or technical collection seems to reinforce the rationale for or against a US policy initiative. In these circumstances, receiving the same "story" from more than one DO or diplomatic source does not in itself speak to the authenticity of the information.
The effort needed to meet the DI standard of ensuring appropriate protection against deception at the lowest net cost to timeliness will vary from case to case. For each assignment, analysts and managers will have to weigh the costs to credibility of being deceived against the opportunity costs of increased care. Below are general schemes for proceeding efficiently against two distinct levels of risk for DI assessments on complex issues.
Regular Issues. We recommend a two-step defense against deception for assessments on which there is no specific reason to suspect deception is at play and on which the primary value added to consumers is a general increase in understanding (as contrasted with tailored support for decision and action):
Step One. Organize the key information by setting it out, for example, on a yellow pad or blackboard, and examine it critically with warning signs of deception in mind. Are all the expected collection systems kicking in? Is the information appropriately consistent? Are the pieces of information on which conclusions and judgments are most heavily based from a reliable clandestine source or uncompromised collection platform?
Step Two. Play Devil's Advocate and develop a hypothetical argument for the case that deception is taking place; that is, a foreign entity is attempting to manipulate perceptions through some combination of denial and disinformation. Then determine to what extent the information organized in Step One lends support to the case for the existence of a deception operation. Look for "hits and misses" between the information and the hypothetical case that substantially raise or diminish the likelihood of the presence of an elaborate deception effort.
Suspect and Sensitive Issues. Where there is reason to suspect the presence of deception (based on general warning signs and the exercises outlined above), analysts should undertake a more elaborate defense of the integrity of their assessments. With or without cause for suspicion, an extended defense should also be employed on sensitive issues - those on which the policy officials may directly rely in making decisions on whether to take military, diplomatic, or economic actions to defend US interests.
On suspect and sensitive issues, we recommend that analysts prepare a textbox or annex that addresses the possibility that a deception operation is distorting the assessment's conclusions and estimative judgments. Managers should consider preparation and defense of the textbox as an essential element of the analyst's effort, even in instances when they determine it is unnecessary to publish it.
The content of the recommended assessment of the integrity of information will vary with circumstances. But at a minimum it should convey that (1) the possibility of the presence of deception has been taken seriously, (2) analytic tests to determine the likelihood of deception have been executed, and (3) any reasonable doubts are forthrightly reported.
Specific questions that analysts can address with profit include:
In sum, such a textbox provides the analysts with an answer to a question likely to be posed with greater frequency by Agency leaders and policy officials in the wake of the Ames case: How do you know you are not being deceived?
Return to "Compendium" Contents