The evaluation of the effectiveness of surveillance technology in intelligence agencies and oversight bodies is notably lacking. Assessments of surveillance technology concerning legal compliance, cost, and matters of privacy occupy a solid place, but effectiveness is rarely considered. Bureaucracy may explain this absence. Applying James Q. Wilson’s observations on bureaucracy reveals that effectiveness is minimally treated because it is more difficult to evaluate than budget assessments and legal compliance, and because intelligence outcomes are unobservable and difficult to oversee. Effectiveness evaluation is thus fettered by bureaucracy. Considerations of bringing in effectiveness assessment must appreciate the realities of bureaucratic constraints to be successful.

Purpose: This study aims to explore how the public perceives the effectiveness of surveillance technology, and how people’s views on privacy and their views on effectiveness are related. Likewise, it looks at the relation between perceptions of effectiveness and opinions on the acceptable cost of surveillance technology. Design/methodology/approach: For this study, surveys of Dutch students and their parents were conducted over three consecutive years. Findings: A key finding of this paper is that the public does not engage in a trade-off neither with regard to privacy-effectiveness (exchanging more effectiveness for less privacy and vice versa) nor with effectiveness-cost, but rather expects all three elements to be achieved simultaneously. This paper also found that the correlation between perceived effectiveness and perceived privacy was stronger for parents than for students. Research limitations/implications: Participants for this study were exclusively in The Netherlands. Survey questions on the effectiveness of surveillance technology focused on one type of technology, and on private mobile device use in two scenarios. Social implications: The public’s perceptions of the effectiveness of surveillance technology potentially influence its acceptance of the technology, which, in turn, can affect the legitimacy and use of the technology. Originality/value: Within the much-discussed privacy-security debate lies a less-heard debate – that of the effectiveness of the surveillance technology in question. The public is one actor in this debate. This study examines the public’s perceptions of this less-heard debate.

Intelligence agencies routinely use surveillance technology to perform surveillance on digital data. This practice raises many questions that feed a societal debate, including whether the surveillance technology is effective in achieving the given security goal, whether it is cost-efficient, and whether it is proportionate. Oversight bodies are important actors in this debate, overseeing budgets, legal and privacy matters, and the performance of intelligence agencies. This paper examines how oversight bodies evaluate the questions above, using documents produced by American and British oversight mechanisms.

In recent years, Western governments have come under sharp criticism for their use of surveillance technology. They have been accused of sweeping up massive amounts of information without evidence of the technologies being effective in improving security. The view of critics is clear, but what do intelligence officials themselves say? This paper analyzes statements of intelligence officials in the U.S. and U.K. from 2006 to 2016, examines what criteria officials use in their discourse on effectiveness, and investigates how considerations of cost and proportionality factor into the equation. It identifies seven measures of effectiveness in the statements of intelligence officials, and finds that cost, though rarely discussed, is the driver behind formalized evaluations of surveillance programs.