Content moderation is a vital condition that online platforms must facilitate, according to the law, to create suitable online environments for their users. By the law, we mean national or European laws that require the removal of content by online platforms, such as EU Regulation 2021/784, which addresses the dissemination of terrorist content online. Content moderation required by these national or European laws, summarised here as ‘the law’, is different from the moderation of pieces of content that is not directly required by law but instead is conducted voluntarily by the platforms. New regulatory requests create an additional layer of complexity of legal grounds for the moderation of content and are relevant to platforms’ daily decisions. The decisions made are either grounded in reasons stemming from different sources of law, such as international or national provisions, or can be based on contractual grounds, such as the platform's Terms of Service and Community Standards. However, how to empirically measure these essential aspects of content moderation remains unclear. Therefore, we ask the following research question: How do online platforms interpret the law when they moderate online content? To understand this complex interplay and empirically test the quality of a platform's content moderation claims, this article develops a methodology that facilitates empirical evidence of the individual decisions taken per piece of content while highlighting the subjective element of content classification by human moderators. We then apply this methodology to a single empirical case, an anonymous medium-sized German platform that provided us access to their content moderation decisions. With more knowledge of how platforms interpret the law, we can better understand the complex nature of content moderation, its regulation and compliance practices, and to what degree legal moderation might differ from moderation due to contractual reasons in dimensions such as the need for context, information, and time. Our results show considerable divergence between the platform's interpretation of the law and ours. We believe that a significant number of platform legal interpretations are incorrect due to divergent interpretations of the law and that platforms are removing legal content that they falsely believe to be illegal (‘overblocking’) while simultaneously not moderating illegal content (‘underblocking’). In conclusion, we provide recommendations for content moderation system design that takes (legal) human content moderation into account and creates new methodological ways to test its quality and effect on speech in online platforms.

Content moderation is a vital condition that online platforms must facilitate according to the law and to create (adequate) online environments for their users. While new regulatory requests, like the Digital Services Act in the European Union create novel obligations for platforms, other legal dimensions like the law of Member States are an additional layer of legal grounds for the moderation of content and relevant for the decisions taken on a day-by-day basis. The decisions taken are either grounded in reasons stemming from law or can be based on contractual grounds like the platform’s Terms of Service and their Community Standards. How to measure these essential aspects of content moderation empirically however is still unclear. We therefore ask the following research question: How do online platforms interpret the law when they moderate online content?To understand this complex interplay and to test the quality of the platform’s content moderation claims empirically, this article develops a methodology that facilitates empirical evidence on the individual decisions taken per piece of content, while highlighting the subjective element of content classification by human moderators. We then apply this methodology to a single empirical case, an anonymous medium-sized German platform which has provided us access to their content moderation decisions. By better understanding how platforms interpret the law we can understand how complex content moderation, its regulation and compliance practices are, as well as to what degree legal moderation might differ from moderation due to contractual reasons in dimensions like the need for context, information, or time.Our results show a considerable divergence between the platforms interpretation of the law and our own interpretation. We believe that a significant number of platform legal interpretations are incorrect. These divergent interpretations of the law mean that we believe platforms are removing legal content that they falsely believe to be illegal (‘over-blocking’) while simultaneously not moderating illegal content (‘under-blocking’). In conclusions, we provide recommendations for content moderation system design that takes (legal) human content moderation into account and creates new methodological ways to test their quality and effect on speech on online platforms.

All platforms conduct content governance, by humans and by algorithms. Especially in times of growing dangers of Corona-related disinformation and a polarised US election campaign platforms have become more active in governing speech. The number of platforms transparency reports and content governance disclosures – voluntarily (like in the US) or not (like in Germany) – is growing. Platforms, however, do not usually include examples in their reports. It is therefore very difficult for researchers to replicate the deletion/non-deletion decisions and check deleted content against national law and platform rules systematically. Both the data (deleted contents) and a taxonomy of norms under which deletion happens have been unavailable. This paper, for the first time, provides a legal and terms of service taxonomy that allows for the evaluation of the quality of individual content governance decisions for German and Austrian online forums. It allows other researchers, when they have access to deleted platform content, to code them systematically and in doing so to better assess the extent to which content governance matches the legal and terms of service categories platforms claim to be implementing.