Estimating crowd density and their emotions for city events using social media images

Abstract

Event managers at large city events use crowd density as a metric in the process of maintaining safety at large scale city events. Identifying the density of crowds at these events relies on expensive physical infrastructure to work well or have limited accuracy. We propose amethod that addresses the gap of not relying on physical infrastructure while incorporating new data features that may help improve accuracy. In addition, Emotion Estimation may prove to be useful for an outlier detection system, such as stampedes. The gap addressed for Emotion Estimation is determining the distribution of emotions shown through facial expressions in social media images at city events. To fill the gap, we propose a new density estimation method and analyze the distribution of emotions at events under normal circumstances. The proposed density estimation method does not rely on physical infrastructure. It estimates the density of crowds using social media images, the images’ location, direction, the number of people in the image, and the image angle of view. We also propose a method to extract the images’ location using Structure fromMotion and Generalized Procrustes Analysis. We analyzed the location estimation through an experiment using manually gathered data. Density estimation was analyzed through an experiment using crowd simulation. Emotion estimation was done through social media images gathered at Kingsday and Sail in the Netherlands originally gathered by Gong et al. [15]. The results show that the location estimation method correctly determines 5% of images that are taken at the event area and 100% of images that are not taken at the event area. We found 5/7 results to be within 100 meters of their true location, according to our findings. For our density estimation technique, the proposed method outperformed other methods that do not rely on physical infrastructure for crowd densities of 0.1-0.3 (People/m2) and social media activity of 0.01-0.03 (images/person). For our emotion estimation, we found that 25% of faces in social media images taken at the examined events were neutral, 70% were happy, and 5% were one of the following expressions: sad, surprised, and fear. No angry facial expressions were found. Our research provides new approaches to calculate the location of social media images and estimate crowd density, which outperformed existing methods. Besides, we provide insights into emotions displayed in social media images taken at city events.