You See It, They Don’t: An Exploratory Study of User-to-User Variation in Instagram Comments
Brahmani Nutakki, Manon Lilott Kempermann, Ingmar Weber · 2026
Read You See It, They Don’t: An Exploratory Study of User-to-User Variation in Instagram Comments by Brahmani Nutakki, Manon Lilott Kempermann, Ingmar Weber (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21953.txt.
Introduction
Nutakki and co-authors test whether Instagram shows different users different comments under its new AI ranking system, announced in March 2025. They ran four fake accounts that varied in gender and political leaning. The accounts collected visible comments on ten news and ten non-news profiles, from two VPN locations. Against their expectations, comments on news posts varied less across users than comments on non-news posts. Account size measures explained the variation better than user traits did. The team offers its collection code for larger audits.
Catalog record
| CREATOR | Brahmani Nutakki, Manon Lilott Kempermann, Ingmar Weber |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29311 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21953
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
You See It, They Don’t: An Exploratory Study of User-to-User Variation in Instagram Comments Brahmani Nutakki, Manon Lilott Kempermann, Ingmar Weber Abstract In March 2025, Meta announced a new AI system to rank the order of the comments shown to Instagram users. With existing research showing how feed personalization systems can lead to increased polarization, the introduction of this new system raises similar questions. This paper presents a small-scale exploratory study examining whether the ranking system produces systematic differences in visible comments shown to different users, particularly for news-related content. Using four sock-puppet accounts varying in gender and political leaning, we collect visible comments on posts from ten news and ten non-news accounts. This collection is repeated twice from two VPN locations to assess location effects. We ask 1) how many visible comments vary across different users, 2) is this variation higher for news accounts than non-news accounts, and 3) can user-attributes like gender, political leaning, and location systematically explain the observed variation. Contrary to our expectations, we find that visible comments on news posts are less likely to vary across users than those on non-news posts. Variation is better explained by account metrics like comment and follower counts than by user attributes. These findings provide an initial glimpse into personalized comment ranking on Instagram and motivate larger, more systematic audits of how comment personalization may shape online discourse. To support further research, we provide the code to collect comments and the data upon request. This work has been accepted at the International AAAI Conference on Web and Social Media Understanding the World Through the Web (ICWSM) 2026 as a poster. Once available, please cite the peer-reviewed version. 1 Introduction Over time, social media platforms have come to function as public squares, where users can engage in discussions on diverse topics. One such extremely popular platform is Instagram, with reports finding that the app is installed on almost 80% of smartphones worldwide, amounting to 3 billion Monthly Active Users ( 7 ) . As early as 2016, Instagram revealed that an average of 95 million photos (and videos) were shared per day, a figure that has likely grown since ( 18 ) . Yet, surveys indicate that only 20% of users actively post on the platform ( 12 ) , implying that most users interact primarily by liking, sharing, or commenting on existing posts. Of the above metrics, comments emerge as the epicenter of dialogue and engagement on the platform: even posts from accounts with 1000-2000 followers receive at least 1-3 comments on average ( 21 ) . This ratio grows drastically, particularly with larger accounts, which can garner tens of thousands of comments. Previous studies have shown that attention-grabbing comments that are highly engaging are frequently negative and emotionally charged ( 19 ; 10 ) . These comments can reduce a post’s credibility ( 15 ) , and a user’s intent to share it ( 2 ) . Conversely, user-initiated corrections can also effectively counter misinformation if they include reliable sources ( 20 ) . In March 2025, Meta announced that an AI system was being used to personalize comments shown to Instagram users ( 14 ) . This system takes different input signals, such as the likelihood of a user to report, delete, reply, click on, or scroll past a comment, to decide how they should be ranked ( 14 ) . As of writing this paper, Instagram lets users choose among three settings for the comments: ‘For You’, ‘Most recent’, and ‘Meta verified’, with ‘For You’ being the default selection. Given existing literature on how social media’s algorithmic personalization may contribute to increased polarization ( 17 ; 4 ) , this announcement has raised similar questions. While there are works that question whether personalization is the primary driver for the observed increase in polarization ( 8 ) , and the long-standing debate on the impact of personalization ( 5 ; 6 ) , recent intervention experiments identified causal links between them ( 16 ) . Exposure to diverse political content can even trigger “backfire effects” increasing polarization ( 1 ) . Prior research also indicates that comments appearing at the top can shape the narrative that follows ( 15 ; 22 ) . Yet, compared with the extensive literature on feed ranking, there appears to be relatively limited research on comment ranking. In this context, we wanted to understand how this new ranking system might influence narratives and potentially amplify polarizing perspectives. Given that this is a relatively new development and conducting large-scale sock-puppet audits is challenging, we begin with a small-scale exploratory study. For this study, our goal is to examine how visible comments vary across users viewing news and non-news content. Nearly 55.8% of Instagram users use the app to stay up-to-date with news ( 7 ) , and news is often polarizing. Social media news comment sections have also been characterized as real-time barometers of public opinion ( 11 ) , with research showing that users who are typically highly engaged with news are more likely to actively comment on the posts ( 13 ) . Given this, news acts as a natural starting point to assess whether user attributes, such as gender, political leaning, and location, affect the comments they see. To provide a baseline, we compare this against non-news content. Our research questions include 1) To what extent do comments vary across users viewing the same post? 2) Is user-user variation greater for news-related posts than non-news posts? and 3) Are user attributes associated with systematic differences in the comments shown? To do this, we analyze visible comments displayed to four sock-puppet users with different political leanings and gender profiles: Female Democrat, Female Republican, Male Democrat, and Male Republican. For each, we collect the visible comments (visible without scrolling, referred to as comments from now on) shown on the same posts from ten news accounts and ten non-news accounts. To test location effects, we repeat the collection from two VPN endpoints in New York State and Texas. We selected these two diverse US locations as most of the news accounts are US-based. Contrary to our initial hypothesis, our results show that, on average, only 12% of the comments differ across users. Comments on news posts are actually less likely to vary than those on non-news posts. Account metrics such as follower and comment counts are highly associated with variation but in opposite directions. By comparison, user attributes have only modest effects and do not appear to be the primary predictors. To our knowledge, this is the first study to investigate personalized comment ranking on Instagram in this context. We hope this exploratory study highlights the need for more systematic research on this development. As platforms increasingly introduce personalization to boost engagement, understanding how these decisions shape discourse and their impact on an already fragmented society becomes vital. Both the code used to collect comments and the data itself are available upon request. 2 Data Collection The data collection pipeline consists of three main steps: (1) creating sock-puppet user accounts, (2) collecting recent posts from selected Instagram accounts, and (3) collecting the top visible comments for each post as viewed by the sock-puppet