Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation DOI: XXXXXXX.XXXXXXX Conference: Under review, 2024; ;

Ruijie Xi, He Ba, Hao Yuan, Rishu Agrawal, Arul Prakash · 2025

Read Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation DOI: XXXXXXX.XXXXXXX Conference: Under review, 2024; ; by Ruijie Xi, He Ba, Hao Yuan, Rishu Agrawal, Arul Prakash (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16065.txt.

Introduction

Ruijie Xi and six co-authors present a study from twenty twenty-five on making search on Facebook Marketplace sharper with fake training data. Search there matches a query to goods by meaning, not just by words. Log data from real use lacks the breadth such models need. The team used large language models to craft fake queries and listings at huge scale.

They tried eight Llama models on a vast set of log points. Fake data came from three routes: new queries, richer listings, and queries from those listings. Models trained on fake data alone often beat those trained on real logs. Mixed sets also gained. Gains in a key score reached a few points, with low rates of made up facts.

Catalog record

CREATORRuijie Xi, He Ba, Hao Yuan, Rishu Agrawal, Arul Prakash
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-16065

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Aug2Search: Enhancing Facebook Marketplace Search with LLM-Generated Synthetic Data Augmentation DOI: XXXXXXX.XXXXXXX Conference: Under review, 2024; ; Ruijie Xi, He Ba, Hao Yuan, Rishu Agrawal, Arul Prakash Abstract . Embedding-Based Retrieval (EBR) is an important technique in modern search engines, enabling semantic match between search queries and relevant results. However, search logging data on platforms like Facebook Marketplace lacks the diversity and details needed for effective EBR model training, limiting the models’ ability to capture nuanced search patterns. To address this challenge, we propose Aug2Search, an EBR-based framework leveraging synthetic data generated by Generative AI (GenAI) models, in a multimodal and multitask approach to optimize query-product relevance. This paper investigates the capabilities of GenAI, particularly Large Language Models (LLMs), in generating high-quality synthetic data, and analyzing its impact on enhancing EBR models. We conducted experiments using eight Llama models and 100 million data points from Facebook Marketplace logs. Our synthetic data generation follows three strategies: (1) generate queries, (2) enhance product listings, and (3) generate queries from enhanced listings. We train EBR models on three different datasets: sampled engagement data or original data ((e.g., “Click” and “Listing Interactions”)), synthetic data, and a mixture of both engagement and synthetic data to assess their performance across various training sets. Our findings underscore the robustness of Llama models in producing synthetic queries and listings with high coherence, relevance, and diversity, while maintaining low levels of hallucination. Aug2Search achieves an improvement of up to 4% in ROC_AUC with 100 million synthetic data samples, demonstrating the effectiveness of our approach. Moreover, our experiments reveal that with the same volume of training data, models trained exclusively on synthetic data often outperform those trained on original data only or a mixture of original and synthetic data. 1. Introduction Facebook Marketplace is a global e-commerce platform that facilitates interactions between buyers and sellers worldwide. With hundreds of millions of products available, the platform supports a vast and diverse array of listings, encompassing everything from furniture and electronics to automobiles and rental properties. This scale, along with the variety of products, poses unique challenges for the search system, which must efficiently match user queries with relevant listings to create a seamless shopping experience. As such, the Marketplace search engine requires advanced retrieval capabilities to capture the complexity and diversity of user queries, linguistic nuances, and product details. Figure 1 . Overview of Aug2Search: For each Facebook Marketplace listing, N synthetic search queries are generated using a GenAI model (Llama). They are both fed to the two-tower SparseNN models for representation learning. In addition, we also enhance product listings with GenAI models. Facebook Marketplace has adopted embedding-based retrieval (EBR) models to enhance search by bridging the semantic gap between user queries and product listings. EBR represents queries and listings as dense embeddings in a shared space, enabling accurate, context-sensitive retrieval despite differences in vocabulary and phrasing ( Huang et al. 2020 ) . Advances like Que2Search have improved query and document understanding efficiency ( Liu et al. 2021 ) , while Que2Engage focuses on surfacing relevant and engaging products ( He et al. 2023a ) . MSURU leverages weakly supervised data for large-scale image classification ( Tang et al. 2019 ) , and HierCat provides hierarchical query categorization to handle Marketplace’s diverse queries with weak supervision ( He et al. 2023b ) . These models collectively strengthen EBR in Marketplace. However, challenges persist around data quality variability and the need to adapt to evolving user behavior. Traditional training data lacks the diversity and detail needed to capture the nuances in user search queries and product listings, limiting the models’ ability to perform accurate semantic matching. For example, product listings provided by sellers often lack standardization, contain typographical errors, and vary widely in style and detail. Moreover, reliance on historical data introduces potential biases that can prevent the model from effectively adapting to emerging product categories and capturing shifts in user search behavior. To address the challenges of data variability, we introduce a flexible and adaptive framework, named Aug2Search, to enhance EBR models through synthetic data augmentation. Figure 1 outlines the Aug2Search framework- which leverages large, pre-trained GenAI models to generate synthetic queries and enhance product listings, capturing the diversity and complexity of real-world search interactions. Aug2Search incorporates a two-tower, multimodal, multistage, and multitask approach to optimize query relevance and search performance, adapting an XLM encoder ( Conneau and Lample 2019 ) to process the query and the text from the listing. The model integrates a shared Multi-Layer Perceptron (MLP) and deep sets fusion to create an image modality token by adapting a transformer-fusion mode ( Yu et al. 2022 ) . To evaluate Aug2Search, we first assess the quality of the GenAI-generated synthetic data through standard metrics and human evaluations. We experiment with and compare eight GenAI models to assess their ability to produce high-quality synthetic data. Next, we evaluate the performance of the EBR model by training on original data, synthetic data, and a combination of both. Our findings confirm the robustness of Meta’s GenAI models, particularly Llama3 and Llama3.1, in generating synthetic queries and listings with high coherence, relevance, and diversity, while maintaining low levels of hallucination. Results indicate that Aug2Search improves ROC_AUC by up to 4% with 100 million synthetic data points. In addition to this, our experiments show that increasing the volume of training data consistently enhances EBR performance, with models trained solely on synthetic data often outperforming those trained on a mix of original and synthetic data. Figure 2 . Framework of Aug2search. 2. Related Work Information Retrieval in E-Commerce Information retrieval (IR) in e-commerce often employs embedding-based methods to align query and item spaces, thereby enhancing retrieval accuracy ( Li et al. 2023 ) . Generative retrieval models have demonstrated strong performance on large benchmarks ( Zeng et al. 2024 ) . Multi-level matching networks bridge the gap between informal queries and categories, boosting classification ( Yuan et al. 2023 ) . Moreover, EBR models have been adopted in e-commerce search to retrieve semantically relevant products as a complement to lexical retrieval ( Li et al. 2021 ; Magnani et al. 2022 ) . HierCat utilizes multi-task learning to handle intent ambiguity and data imbalance, improving relevance and engagement ( He et al. 2023b ) . Que2Search and Que2Engage optimize multimodal and multitask query-product representation and balance relevance and engagement ( Liu et al. 2021 ; He et al. 2023a ) . Recent advances in reranking and truncation integrate these tasks, achieving state-of-the-art results ( Xu et al. 2024b ) . Generating Synthetic Data for E-Commerce Systems Synthetic data generation enhances e-commerce retrieval and query coverage. Large language models (LLMs) generate synthetic queries to improve responsiveness and capture complex user queries ( Rome et al. 2024 ; Sannigrahi et al. 2024 ) . In e-commerce, LLM-generated queries align better with user intent, as evidenced by increased Click-Through Rates (CTR) in applications such as Alipay ( Wei et al. 2024 ) . They also expand search capabilities for new categories, bridging semantic gaps in long-tail queries ( Li et al. 2022 ; Jagatap et al. 2024a ) . Additionally, LLMs support product attribute extraction and increase relevance in listings ( Fang et al. 2024 )

Read the complete text