Semantic insurance pricing with large language models

Christopher Blier-Wong, Derek Kusmenko · 2026

Read Semantic insurance pricing with large language models by Christopher Blier-Wong, Derek Kusmenko (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29371.txt.

Introduction

Blier-Wong and Kusmenko test whether language-model embeddings can replace hand-built features in insurance pricing. Classical actuarial models such as the generalised linear model are transparent and easy to govern, but they capture interactions among risk factors only when analysts engineer those features by hand. The pair instead describe each policyholder in natural language, embed that text with a pre-trained language model, and feed the embeddings into a standard pricing model, mainly Poisson regression for claim frequency.

On the French motor third-party liability dataset, the embedding-based model beat the plain generalised linear model, especially when data were scarce. At larger samples the comparison depended on model and dimension. Fine-tuning the language model on insurance text improved the embeddings further. A diagnostic showed the pipeline reacts to any extra appended field, so the authors argue controlled prompts are a governance requirement.

Catalog record

CREATORChristopher Blier-Wong, Derek Kusmenko
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.29371

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Semantic insurance pricing with large language models Christopher Blier-Wong, Derek Kusmenko Abstract Classical actuarial pricing models, such as the generalized linear model, are valued for transparency and ease of governance, but they use interactions among risk factors only when these are supplied through explicit feature engineering. We study whether embeddings from a pre-trained large language model, computed from a natural-language description of each policyholder, can replace hand-crafted features as inputs to a standard actuarial pricing model, taking Poisson claim-frequency regression as the main example. The language model is used only to construct deterministic embedding covariates; pricing is performed by a standard generalized linear model. Using French motor third-party liability data, the embedding-based model outperforms the generalized linear model, especially when data are scarce, whereas at larger sample sizes the comparison is model- and dimension-dependent. Insurance-specific fine-tuning further improves the embeddings, and a prompt-sensitivity diagnostic shows that the pipeline reacts to any appended out-of-template field, making controlled prompts a governance requirement. Keywords : Insurance pricing, risk classification, underwriting, representation learning, large language models, embeddings 1 Introduction In motor insurance pricing, actuaries use variables such as driver age, vehicle characteristics, geography, and claim history to estimate claim frequency. In non-life insurance, a range of predictive models is used for this purpose, including generalized linear models (GLMs), tree-based and boosting methods, neural networks, and hybrid architectures such as the Combined Actuarial Neural Network of 58 ; GLMs remain attractive in practice for their transparency and ease of governance. In this paper, we focus on claim frequency. A standard actuarial approach is to fit a Poisson GLM that relates claim frequency to structured covariates such as driver age, vehicle type, region, and bonus-malus level through a log-link function. Because GLMs assume a linear relationship between the covariates and the log-expected claim count, capturing interactions between risk factors requires explicit feature engineering: the actuary must identify which pairs or groups of variables interact and construct the appropriate interaction terms. A GLM without interaction terms may understate risks that appear only in combinations: an 18-year-old driver with a high-powered vehicle in a dense urban area may present a risk profile that is not well represented by separate additive effects for age, vehicle power, and location. Flexible learners such as gradient-boosted trees or neural networks can learn such patterns from data when sufficient observations are available, see 26 for a comparison study. The question we study is whether an embedding representation can make similar structure usable inside a GLM, especially at small sample sizes. Recent actuarial work has replaced one-hot or hand-crafted variables with learned representations of categories, locations, text, and images. Embedding-based representations of high-cardinality categorical covariates were introduced to actuaries by 45 ; 46 . These ideas have been extended in ( 48 ; 12 ; 2 ; 44 ) . However, these approaches require neural networks to be trained jointly with the pricing model, which may be challenging when the dataset is small or the signal-to-noise ratio is low. The representation learning framework of 8 decouples the feature construction step from the prediction step: a representation model transforms raw data into dense vectorial embeddings, and a standard GLM then uses these embeddings as features. This framework has been applied to incorporate geographic data ( 9 ; 27 ) , textual claims data ( 59 ) , and image data ( 10 ) into insurance pricing models. In these applications, embeddings compress sparse inputs, such as geographic grids, claim notes, or images, into fixed-length vectors that can be used in a GLM. In parallel, large language models (LLMs) built on the transformer architecture of 51 produce internal representations that reflect semantic relationships between concepts, including factual knowledge, domain-specific associations, and contextual dependencies learned from massive text corpora ( 7 ; 43 ) . Recent tabular-LLM studies serialize rows as text and often report the largest gains when labeled samples are scarce (see, e.g., ( 23 ; 54 ) ). A related line of work extracts embeddings from LLMs applied to serialized tabular data and uses these as features for a separate supervised model, separating representation from prediction (see, e.g., ( 52 ; 34 ; 32 ) ). We review this literature in detail in Section 2.5 . In this paper, we propose using pre-trained LLM embeddings to replace traditional hand-crafted features in the actuarial pricing pipeline. We use the embedding model as a fixed reparameterization of the variables: the resulting vector contains no new fields, but may make some interactions easier for a generalized linear model to use. We do not manually specify any interactions; instead, we test whether embeddings of the serialized policyholder descriptions contain directions that improve a downstream GLM, and we fine-tune the embedding model using low-rank adaptation (LoRA, 28 ) and a ranking-based contrastive objective that aligns the embedding geometry with claim frequency similarity. Two properties of the representation motivate this. First, the embedding model maps categorical covariates such as region and vehicle brand to dense vectors that may reflect similarities among category labels rather than treating each level as an isolated one-hot indicator. Second, because self-attention combines information across token positions, the embedding can depend on combinations of fields and not only on each field separately. This motivates testing whether the embeddings carry interaction-relevant variation that a GLM could use; whether either property is useful for claim-frequency pricing is an empirical question. We evaluate the approach across training set sizes from a few hundred to several hundred thousand policies, using a Poisson GLM with exposure offset as the downstream model, and we fine-tune the embedding model on an insurance-specific task with LoRA; Section 2.5 positions these contributions relative to prior work. As a supplementary check (Appendix A ), we also probe whether the embedding pipeline reacts to tokens that should not influence pricing. The remainder of this paper is structured as follows. Section 2 provides the technical background on transformer architectures, embedding models, fine-tuning, and the connection between embeddings and actuarial pricing. It also reviews related work. Section 3 describes the prompt design, embedding extraction, and downstream pricing model. Section 4 presents the data and experimental setup. Sections 5 and 6 report the off-the-shelf and fine-tuned embedding results, respectively, and Section 7 discusses practical considerations and limitations and concludes. We also prepare two appendices to further interpret how the methodology works. Appendix A a protected-attribute prompt sensitivity diagnostic, and Appendix B an unsupervised clustering of the embedding into rating bands. 2 Background 2.1 The transformer architecture Modern large language models are built on the transformer architecture ( 51 ) , which converts an input string into a sequence of tokens and refines a vector representation of each token through a stack of self-attention and feed-forward layers. The property relevant here is that self-attention lets each token’s representation depend on the other tokens in the input, so a field such as a driver’s age can in principle be encoded differently depending on the vehicle and location fields that accompany it. A GLM with additive terms cannot represent such compounded structure unless the relevant interaction terms are supplied explicitly; whether the embedding makes this structure usable for claim-frequency pricing is an empirical question that the experiments below address. In our pipeline, the

Read the complete text