10th Biennial ACSPRI Social Science Methodology Conference in conjunction with the Australian Social Network Analysis Conference (ASNAC)

Social media is now pervasive in modern society, and is used not only for entertainment purposes, but has become a central environment for political and social discourse. Understanding, and being able to measure and identify, when online conversations are productive and contribute positively to public discourse, is increasingly important in order to limit toxic and hostile behaviour in a partisan society.

We propose a novel multi-layer network model to study deliberation in online discussion platforms. The model comprises two layers: a discussion layer, which represents as a hierarchical tree the comment-to-comment replies within a given post, and an actor layer, which represents the actor-to-actor reply interactions. The interlayer links represent user-comment ownership. We use this network model to extend Gonzalez-Bailon et al. (2010)’s foundational work on using network analysis to measure argumentation and representation in discussion, by proposing two network metrics to measure the potential for deliberation in discussion threads. We also propose a new approach for identifying the boundaries that determine whether a given post is identified as deliberative, and for comparing the level of deliberation of collections of posts.

We apply our new approach to a large Reddit dataset containing over 200,000 posts from 72 subreddits focused on different topics and demonstrate that our new metrics have construct validity in that observed differences in measured deliberation across categories of subreddits can be explained by the topics and purpose of the subreddits. While we find that some political subreddits exhibit high levels of deliberation thus indicating that online political talk can meet the prerequisites for deliberation, the level of deliberation in regional and sports subreddits was even higher. Analysis of the actor layer shows correlations between the deliberation measured on the discussion layer, and classical network metrics of the actor layer including degree distribution, assortativity, and the global clustering coefficient. These correlations reveal important insights into how the aggregate behaviour of users is related to deliberation.

Finally, we discuss planned future work, including refinement of the metrics by incorporating comment engagement data (such as upvotes) and use of Natural Language Processing methods to identify linguistic markers for deliberative conversations. Ultimately, we hope to identify methods that can be implemented to help promote deliberative conversations in online platforms.