Conceptualising Social Media for Proactive Intelligence Enhancement: An empirical approach to enrich near real time open source intelligence support for large-scale social movements

Abstract

The open-source intelligence (OSINT) team of the Netherlands Police monitors potential threats to public safety. While a variety of open sources are used in preparation for large-scale protests, social media (particularly Twitter) offers unique insights into the real-time dynamics of public opinion, enabling early estimation of potential escalations. This study analyses a large corpus of manually labelled Twitter data, comparing a range of state-of-the-art natural language processing methods. Baseline approaches, including bi-word analysis, n-grams, and support vector machines, are evaluated alongside more advanced models such as BERT, SetFit, and Bernice, with the latter achieving the highest performance in sentiment analysis tasks. Model performance is further enhanced through training on synthetically generated data. The multi- modal nature of tweets allows for user-level social networks, enabling the scrutiny of topic diffusion within online communities. Agent-based simulations are included to investigate emerging social dynamics, such as spill-over effects and polarisation, highlighting opportunities for proactive modelling to predict which topics are likely to gain traction. The resulting models are integrated into a prototype dashboard, designed using a methodology that combines user-centred design principles with ecological interface design. The dashboard, tested with OSINT practitioners, was well received for its speed, usability, and output quality, matching existing tools. The study delivers multiple tangible outputs: (1) four manually labelled datasets of protest-related tweets, developed under the supervision of the Netherlands Police OSINT team and accompanied by detailed labelling protocols; (2) Tweeti, a tool to support the labelling process; (3) a methodology for generating social networks grounded in information needs and societal values; (4) several language models capable of mapping demonstration-related dimensions, including sentiment, topic distribution, geolocation, and social network structure; and (5) an integrated dashboard for synthesising and presenting these insights to end users.