Model in Distress: Sentiment Analysis on French Synthetic Social Media

Pierre-Carl Langlais Pavel Chizhov Yannick Detrois Carlos Rosas Hinostrosa, Ivan P. Yamshchikov Bastien Perroy · 2026

Read Model in Distress: Sentiment Analysis on French Synthetic Social Media by Pierre-Carl Langlais Pavel Chizhov Yannick Detrois Carlos Rosas Hinostrosa, Ivan P. Yamshchikov Bastien Perroy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18226.txt.

Introduction

Langlais and five colleagues generate synthetic French social-media data to study customer distress in public transit. Labeled data is costly, evaluation sets are scarce, and privacy rules block sharing real user posts. Backtranslation from a small seed corpus yields 1.7 million synthetic tweets plus reasoning traces. Reasoners trained on this data reach 77 to 79 percent accuracy on human-labeled tests, matching top proprietary models while exposing no real user data.

Catalog record

CREATORPierre-Carl Langlais Pavel Chizhov Yannick Detrois Carlos Rosas Hinostrosa, Ivan P. Yamshchikov Bastien Perroy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39604
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.18226

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Model in Distress: Sentiment Analysis on French Synthetic Social Media Pierre-Carl Langlais Pavel Chizhov Yannick Detrois Carlos Rosas Hinostrosa, Ivan P. Yamshchikov Bastien Perroy Abstract Automated analysis of customer feedback on social media is hindered by three challenges: the high cost of annotated training data, the scarcity of evaluation sets, especially in multilingual settings, and privacy concerns that prevent data sharing and reproducibility. We address these issues by developing a generalizable synthetic data generation pipeline applied to a case study on customer distress detection in French public transportation. Our approach utilizes backtranslation with fine-tuned models to generate 1.7 million synthetic tweets from a small seed corpus, complemented by synthetic reasoning traces. We train 600M-parameter reasoners with English and French reasoning that achieve 77-79% accuracy on human-annotated evaluation data, matching or exceeding SOTA proprietary LLMs and specialized encoders. Beyond reducing annotation costs, our pipeline preserves privacy by eliminating the exposure of sensitive user data. Our methodology can be adopted for other use cases and languages. DistressedModel/Distressed-Model-Data 1 Introduction Large companies and services handle a substantial volume of reviews and social media mentions. Social media has become a primary means of filing complaints due to the need for unfiltered, independent feedback and ease of use 44 ; 12 ; 29 . This convenience has driven a corresponding increase in the volume of complaints. For example, by 2014, complaints in the United Kingdom had doubled compared to 2013, with 31% expressed through social media 27 . Social media also provides anonymity and eliminates direct confrontation, enabling customers to express frustration while exerting perceived public pressure that private reviews may lack 7 . Furthermore, if customers complain offline or in private, they may then opt for a second round of complaint online 19 . As customer expectations around social media engagement grow, response time has become critically important 43 . For instance, it was recently shown to influence the chances of repurchase 26 . Therefore, it is crucial for a company to quickly detect the issues and react accordingly by implementing a service recovery 25 . When services operate at scale, and social media interactions exceed manual processing capacity, automated tools such as language models offer a practical solution. Language models of different sizes and with different design purposes are being developed daily. Proprietary large language models (LLMs) might act as agents for such purposes, while an alternative solution would be specialized small language models. The benefits of LLMs typically include their versatility and ability to understand complex contexts, while specialized small language models are favored for their robustness and, what is especially relevant to public institutions, for the traceability of open-source solutions and an opportunity for autonomous deployment. A crucial part of automatic review evaluation is sentiment analysis, which typically involves detecting a wide range of emotions and distress to identify the purpose and sentiment of the user messages and classify them into categories required by the business. The models for such a classification task require training on a substantial portion of representative data and reliable annotations, which are often costly to obtain. Furthermore, the trained models need to be properly evaluated, which is often problematic due to the specificity of each case and the lack of comprehensive benchmarks, especially for non-English languages. Even though modern machine translation systems are very advanced, simply translating all the training data into the required language would degrade data quality 6 , especially given the domain’s complex style. Synthetic data generation has recently emerged as a promising method for training specialized models in regulated sectors with the lack of annotated data 24 ; 31 ; 33 ; 47 . Pipelines provide privacy and compliance by design, as personal or sensitive data is never exposed; instead, it is substituted with realistic simulations. In this work, we introduce one of the first large-scale synthetic pipelines for customer reviews on social media. The pipeline can be applied across various domains and languages to address the aforementioned challenges. We demonstrate its effectiveness on the example of Paris public transport. Our key contributions include: • We collect a small dataset of tweets annotated by both native speakers and LLMs and aggregate these annotations into reliable labels. • We present a methodology for creating a synthetic pipeline for review generation using backtranslation and synthetic reasoning, which is applicable to other domains, and release a large collection of synthetic tweets 1 1 1 We release the anonymized synthetic tweets on HuggingFace: DistressedModel/Distressed-Model-Data . . • We train small reasoning models using synthetic data and evaluate them using human-annotated data, reaching a performance comparable to or better than that of modern LLMs. 2 Related Work Previous sentiment analysis research was largely concentrated on English data. TweetEval 9 combined several SemEval tasks into a compound benchmark for sentiment analysis 41 , emotion recognition 35 , stance 36 , irony 49 , hate speech 11 , and offensive language detection 52 , as well as emoji prediction 10 . All the subsets were exclusively in English. Despite the initial prevalence of small encoder models applied to this task, it was recently used to evaluate large language models 8 . Previous efforts for non-English data collection and classification included tweet sentiment analysis in 14 African languages 39 , multi-modal analysis in 21 languages 48 , and other approaches in Eastern European languages 18 , Bengali 42 , Indonesian 21 , and Hausa 3 . Encoder-style solutions. Typical solutions for the classification tasks of such sort are small BERT-like models 22 ; 37 , or classical machine learning approaches to building embeddings (TF-IDF, Bag of Words, Word2Vec, etc.) and classification algorithms (Logistic Regression, Random Forest, etc.) 1 , as well as custom pipelines combining both approaches 38 . Decoder-style solutions. There has also been a growing line of work where decoder-style language models are used for sentiment analysis and review classification 45 ; 16 . Even though these approaches are more computationally intensive, which is partially mitigated by recent advancements in small language model development 20 ; 51 , they often provide additional features such as custom prompts 45 and reasoning 2 . 3 Task and Motivation Figure 1: Synthetic pipeline and distress detection model training. Left: auxiliary model fine-tuning. Right: synthetic pipeline for generating synthetic tweets and training the distress detection model. All LLM-generated data are shown in purple . All pipeline-generated synthetic data are shown in pink . In this work, we analyze public transport reviews on Twitter in French and aim to build a distress detection tool that enables support agents to quickly monitor issues expressed on social media. While the task is similar to an ordinary classification pipeline, we approach it in a non-standard way by building a synthetic tweet generation pipeline and training small reasoning models. This is motivated by the following factors: • Imbalance: operational classification of distress is different from the abstract definition and concerns a very small selection of tweets. Using only organic data would put a ceiling on the capabilities of small models. • Data drift: even though a large historical dataset of 30 million tweets collected from 2012 onward (see Appendix A ) could be leveraged to mitigate data scarcity, there have been substantial changes in user expression and

Read the complete text