Researchers at the Pennsylvania State University have developed an artificial intelligence (AI) model capable of detecting extremist users and content related to the militant group Islamic State (ISIS) on social media platforms. Using tweets spanning from 2009 to 2021, the predictive model aims to assist social media companies in identifying and restricting such accounts more efficiently, minimizing their impact on online communities.
The research team, employing machine learning and natural language processing techniques, analyzed tweets, identified potential propaganda messages, and developed an image classifier to categorize images attached to ISIS-related tweets. Their work addresses the ongoing challenge of extremist groups leveraging social media for propaganda dissemination and recruitment.
Younes Karimi, a graduate student at the university and the first author of the paper published in the journal Social Media Analysis and Mining, emphasized the manipulation of online communities by ISIS and its affiliates to spread extremist propaganda. Despite countermeasures implemented by platforms such as X (formerly Twitter), these groups continue to exploit social media for their activities.
The AI model distinguished users sharing ISIS-related content by analyzing machine learning predictions based on past data and manipulating textual data through natural language processing. The dataset included known members of the Islamic State group and individuals who engaged with ISIS-related content by retweeting, quoting, or mentioning it.
The research identified users who retweeted or quoted ISIS content as likely affiliates or sympathizers, while those who merely mentioned the content were considered less likely to be supporters. The team analyzed tweets to identify “candidate propaganda,” comparing topics used by known ISIS accounts before 2015 with content posted after 2015 by potential affiliates and supporters.
Key findings included the pervasive and continuous sharing of identified content, the use of ideology-based words and images to elicit emotional responses, and the strategic use of hashtags to create trending ideas and curate group messaging.
The researchers highlighted the importance of the longitudinal perspective of the dataset, covering data from before and after the 2015 crackdown by Twitter on ISIS-related accounts. The study sheds light on the evolving online strategies of extremist groups and the need for effective measures across various social media platforms.
