Notes on Analytic
This is the sixth in a series of Product Evaluation Staff notes to clarify
the standards used for evaluating DI assessments and to provide tradecraft
tips for putting the standards into practice.
Analytic expertise establishes the DI's authority to speak to an issue on
which US national security interests are at stake. Demonstration of expertise
in DI assessments is thus needed to gain access to and credibility with
the policymakers, warfighters, and law enforcement officials who carry the
heaviest burden for US decision and action on national security issues.
As with all evaluation criteria addressed in these tradecraft notes,
analysts have to tailor their demonstration of expertise in individual memorandums
and briefings to the circumstances of each assignment. These include the
degree of substantive complexity, the analytic requirements and timeline
of the policymaking process, and whether the DI production unit already
has established close ties to its key clients.
As a rule though, no DI product should be delivered without deliberate
effort to make it stand out from the formidable competition including other
intelligence providers, journalists, scholars, lobbyists, and various additional
purveyors of information and judgment:
- Demonstration of in-depth substantive knowledge is usually the most
important element in establishing the credentials for a DI product. To
a large extent, the Directorate is what it knows and can bring to bear
to reduce uncertainty on complex issues.
- Analytic expertise is also conveyed by (1 ) skill in accessing and
interpreting clandestine and technically collected information, (2) deft
organization of data in support of findings and judgments, (3) dealing
effectively with longshot contingencies and with uncertainty generally,
and (4) demonstrating insight into the decisionmaking processes of intelligence
How much expertise needs to be demonstrated? Taking space and time limitations
into account, the more the better. The DI not only has to stand out regarding
the breadth and depth of its analytic expertise, it also has to convince
its key customers that it can put this expertise to use for their benefit.
Again, the appropriate means through which to convey expertise varies
with each analytic assignment. As a general test, we recommend that analysts
read their first draft critically, to answer the following questions as
if they were posed by a policy official deciding whether to rely on the
DI to meet the challenges on his or her agenda.
- Do the analysts show in-depth knowledge of their subject the history,
civil culture, and other general contexts of the issue, as well as the
- Is there evidence that the analysts have "been there" or
represent an analytic and operational team with members who have experience-based
- Have the analysts brought to bear all the dimensions or disciplines
that define the issue, political, economic, military, S&T?
- Have the analysts brought to bear all-source information and made a
special effort to help the busy intelligence user grasp the key patterns
revealed by the information?
- Do the timing and format reflect the analysts' sensitivity to the timeline,
decision points, and underlying tensions (areas under debate) of the policymaking
- Do the analysts have an effective method or structure for arraying
information and reducing uncertainty?
- Further on uncertainty, do the analysts help me think through complex
challenges by making their key assumptions clear and dealing rigorously
with alternative interpretations and outcomes?
- In sum, would I want to rely on DI analysts and pose follow-up tasking,
to deal with the threats to, and opportunities for, US interests on my
Research papers are natural vehicles for effective display of analytic
expertise. DI veterans, over the years, have devised an armory of measures
for demonstrating hard-earned individual and collective expertise even in
quick turnaround papers and within the space limitations for most DI deliverables.
We list below some of these measures:
- Evidence of previous research. In a preface, footnote, textbox,
or annex, list related finished intelligence on the subject at hand. Be
sure to take a Directorate or team approach by citing the works of other
- Evidence of scholarship. Often one footnote or textbox that
indicates the breadth of the analysts' grasp helps to establish credentials.
For example, citing a historical incident with either common or contrasting
elements to the issue at hand.
- Evidence of data bases. Expertise can be demonstrated by a paragraph
or textbox that presents a chronology, a series of quotes over time by
a foreign leader, or a schematic representation of trends (terrorist incidents,
inflation or economic growth rates, growth in military inventories, S&T
- Evidence of ground truth. At times it takes only a sentence
or two to demonstrate special insight into key personalities, groups, and
forces at work. Use your own feel for the ground truth or rely on your
analytic teammates to work in some experience-based references. Examples
could include references to local political humor, to the household commodities
on which short supplies would be most likely to cause popular disgruntlement,
or to the modus operandi of local intelligence and security forces.
- Evidence of teamwork. Put your professional support colleagues
to work on your behalf to clarify a key point or to organize and display
the information supporting a point - via annotated maps, imagery, and other
graphics. Interviews with customers indicate that those who are heavily
engaged on an issue particularly appreciate visual aids that help in keeping
up with the fire hose of information.
- Leadership analysis. US leaders, naturally enough, tend to be
interested in foreign leaders, especially their approach to making decisions,
negotiating, and dealing with conflict. Addressing these matters in the
text, a textbox. or an annex demonstrates both the breadth and depth of
the DI's collective expertise.
- Secrets. Excessive classification should be avoided in assessments
intended for a broad and general audience. But when a DI assessment is
intended for the handful of officials fully engaged (and fully cleared)
on an issue, contextual explanation of information from clandestine and
technical sources conveys a sense of special expertise.
- Mysteries. Demonstrating skill in dealing with uncertainty is
appreciated by the hands-on policy officials who have to decide which risks
to cover. This includes making explicit the assumptions that drive the
argumentation and taking account of low-probability, high-impact contingencies.
Here, too, the coverage can be set aside in a textbox or annex, if the
goal is to keep the main text lean.
- The only game in town. Understandably, policy officials believe
that policymaking and implementation is the reason why intelligence exists,
and not vice versa. Analysts should demonstrate - in a cover note, preface,
or footnote, if not in the main text - that they are informed on the policy
timeline and the key matters under debate. Indeed, few things call the
DI's reputation into question more than an assessment that arrives after
a decision has been made or that addresses in a conventional manner issues
on which there already is both broad knowledge and consensus among policy
- Customized support. When writing for a small number of officials,
analysts can demonstrate by the issues they highlight that they know where
their consumers are on their learning curve as well as their decision cycle.
Expertise in Support of the Intelligence Consumer
We end with an obvious caution. When the analysts know their clients'
specific needs, demonstration of expertise is readily projected as a tool
or means for providing value added With larger assessments prepared for
a broader audience, more care has to be taken to avoid excessive display
of knowledge - that is, substantive expertise as an end in itself. Here
is where investment in knowing the timelines of the policymaking process
and the specific dangers and opportunities policy officials are dealing
with comes into play.
Forward to "Note 7"
Return to "Compendium" Contents