REAL: Benchmarking Abilities of Large Language Models for Housing Transactions and Services
Kexin Zhu, Yang Han · 2025
Read REAL: Benchmarking Abilities of Large Language Models for Housing Transactions and Services by Kexin Zhu, Yang Han (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03477.txt.
Description
Kexin Zhu and Yang Han build REAL, the first test suite for judging whether language models can act as housing agents. Chatbots are spreading into sales and service work, but real estate demands memory, understanding, reasoning, and honesty about facts. The suite holds 5,316 checked questions across those four topics, sorted into fourteen categories. Each entry probes knowledge or skill a housing transaction needs.
The authors ran the leading current language models through the suite. Performance left wide room for growth before models could handle real housing work unaided. Gaps appeared across the board rather than in a single topic. The benchmark gives developers a fixed yardstick for a field where errors carry legal and financial weight. It also shows buyers and firms where today's chatbots still need human backup.
Catalog record
| CREATOR | Kexin Zhu, Yang Han |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03477