Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction
Aishni Parab, Hongjing Lu, Ying Nian Wu, Sumit Gulwani · 2025
Read Analysis of Error Sources in LLM-based Hypothesis Search for Few-Shot Rule Induction by Aishni Parab, Hongjing Lu, Ying Nian Wu, Sumit Gulwani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01016.txt.
Description
Aishni Parab, with three co-authors, analyzes error sources in LLM-based hypothesis search. The task is few-shot rule induction. This kind of reasoning infers abstract rules from few examples. Humans do this well and apply rules to new cases. The team asks how close machine approaches come.
They compare hypothesis search with direct program generation. The first approach proposes explicit rules and tests them. The second writes programs in one pass instead. The first achieves performance comparable to humans. The second falls notably behind on the same tasks. An error analysis reveals key bottlenecks in generating hypotheses. It also suggests ways to advance program induction. The work shows both the promise and the challenges here.
Catalog record
| CREATOR | Aishni Parab, Hongjing Lu, Ying Nian Wu, Sumit Gulwani |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-01016