Evaluating the quality of intelligence analysis: The analytic accuracy of American strategic intelligence estimates regarding the Arab-Israeli conflict between 1947 and 1974

Abstract

Evaluating the quality of intelligence analysis is important for both intelligence consumers (i.e. policy and decision makers), and producers (i.e. intelligence professionals), because consumers’ perception of the accuracy of intelligence products to some degree determines the extent to which the intelligence is heeded. This study tests the utility of using the broad concept of analytic accuracy (including driver accuracy, driver selection, and estimative accuracy) in this regard. It differentiates between levels of accuracy, and considers possible explanations for the degree of analytic accuracy. American strategic intelligence estimates regarding the Arab-Israeli conflict from 1947 until 1974 are used as a case to test the viability of this theoretic framework. The study shows that including drivers in evaluating intelligence products provides a more useful take on accuracy than focusing solely on overall assessments. This gives intelligence organisations, and consumers better and more starting points for improving intelligence analysis, consumer-producer interaction, and intelligence collection and management. Furthermore, it allows intelligence scholars to ameliorate the way they evaluate the quality of intelligence production, and their judgment of intelligence performance.