Evaluating LLM-persona Generated Distributions for Decision-making

Jackie Baek, Yunhan Chen, Ziyu Chi, Will Ma · 2026

Read Evaluating LLM-persona Generated Distributions for Decision-making by Jackie Baek, Yunhan Chen, Ziyu Chi, Will Ma (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06357.txt.

Description

A firm pricing a new product could ask a language model to invent a crowd of fake customers and read off what they would pay. Jackie Baek and 3 co-authors study that strategy, which they call LLM-SAA: build a distribution from model output, then optimize the decision under it. The question is whether such invented data supports good decisions.

They test three classic problems: assortment choice, pricing, and the newsvendor stocking problem. The model-made distributions proved practically useful, especially when real data were scarce. Judging them by plain distance measures, like the Wasserstein distance, could mislead, since a distribution can sit far away yet still induce near-right decisions. Decision-aware metrics are the better yardstick.

Catalog record

CREATORJackie Baek, Yunhan Chen, Ziyu Chi, Will Ma
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-06357

Browse the full library · About the Tower of Babel