WLFM: A Well-Logs Foundation Model for Multi-Task and Cross-Well Geological Interpretation Journal: Geoenergy Science and Engineering

Zhenyu Qi, Qing Yu, Jichen Wang, Yun-Bo Zhao, Zerui Li, Wenjun Lv · 2025

Read WLFM: A Well-Logs Foundation Model for Multi-Task and Cross-Well Geological Interpretation Journal: Geoenergy Science and Engineering by Zhenyu Qi, Qing Yu, Jichen Wang, Yun-Bo Zhao, Zerui Li, Wenjun Lv (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-18152.txt.

Introduction

Zhenyu Qi and 5 co-authors build a base model for reading well logs, the depth traces used to map rock below ground. Their 2025 model, WLFM, faces messy tool signals, noise, and few tags. It trains on logs from more than a thousand wells, with no tags at first.

Log curves are cut into patch tokens that stand for rock units. Masked-token and contrast steps teach the model rock sense in a self-taught way. Few-shot tuning then fits it to tasks such as pore share and rock type. Scores beat top baselines, and the model shows a learned rock word set.

Catalog record

CREATORZhenyu Qi, Qing Yu, Jichen Wang, Yun-Bo Zhao, Zerui Li, Wenjun Lv
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-18152

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

WLFM: A Well-Logs Foundation Model for Multi-Task and Cross-Well Geological Interpretation Journal: Geoenergy Science and Engineering Zhenyu Qi, Qing Yu, Jichen Wang, Yun-Bo Zhao, Zerui Li, Wenjun Lv Abstract Well-log interpretation is fundamental for subsurface characterization but remains challenged by heterogeneous tool responses, noisy signals, and limited labels. We propose WLFM, a foundation model pretrained on multi-curve logs from 1200 wells, comprising three stages: tokenization of log patches into geological tokens, self-supervised pretraining with masked-token modeling and stratigraphy-aware contrastive learning, and multi-task adaptation with few-shot fine-tuning. WLFM consistently outperforms state-of-the-art baselines, achieving 0.0041 MSE in porosity estimation and 74.13% accuracy in lithology classification, while WLFM–Finetune further improves to 0.0038 MSE and 78.10% accuracy. Beyond predictive accuracy, WLFM exhibits emergent layer-awareness, learns a reusable geological vocabulary, and reconstructs masked curves with reasonable fidelity, though systematic offsets are observed in shallow and ultra-deep intervals. Although boundary detection is not explicitly evaluated here, clustering analyses suggest strong potential for future extension. These results establish WLFM as a scalable, interpretable, and transferable backbone for geological AI, with implications for multi-modal integration of logs, seismic, and textual data. 1 Introduction Figure 1: Conceptual motivation of the proposed WLFM framework. Well-log interpretation is challenged by noisy, incomplete, and heterogeneous multi-curve data. Traditional rule-based or task-specific models lack scalability and cross-domain generalization. WLFM introduces a tokenization-based foundation modeling approach that bridges domain-informed priors with large-scale representation learning. Well logging provides high-resolution measurements of subsurface formations and plays a central role in lithology classification, boundary detection, and reservoir characterization [ 12 , 36 , 31 ] . It underpins critical applications including unconventional reservoir evaluation, carbon storage monitoring, and basin-scale geological modeling. However, practical interpretation is often constrained by noisy signals, missing curves, and heterogeneous data distributions across wells, which substantially limit the scalability and generalization of conventional workflows [ 10 , 3 , 4 ] . 1.1 Limitations of Existing Machine Learning Approaches The rise of machine learning has created new opportunities for automating and scaling well-log interpretation. Compared with rule-based approaches, neural networks can process large datasets, capture nonlinear dependencies, and improve predictive accuracy across diverse geological settings [ 30 , 33 , 7 , 19 ] . Early studies employing convolutional neural networks (CNNs) and recurrent neural networks (RNNs) demonstrated promise in lithology identification [ 29 , 32 , 25 , 14 , 24 ] , porosity prediction [ 22 , 9 ] , and thin-bed facies analysis [ 13 ] . Yet, these approaches typically required extensive labeled data and exhibited poor transferability across formations. Domain adaptation strategies, such as maximum mean discrepancy (MMD) [ 6 ] and partial adaptation [ 17 ] , attempted to mitigate cross-well distribution gaps, but relied heavily on heuristic assumptions and showed limited robustness under sparse supervision. More recent adversarial and GAN-based methods [ 26 , 2 , 23 ] enhanced transferability but introduced architectural complexity and remained vulnerable to noise and missing data. 1.2 Toward Foundation Models in Geoscience Beyond well-log analysis, deep learning has demonstrated broader potential in geophysical interpretation. For instance, Tang et al. [ 28 ] applied neural networks to transient electromagnetic time-series for subsurface body prediction, highlighting the value of sequence representation learning in geoscience. In seismic studies, foundation-style architectures have been applied to inversion [ 35 , 34 ] , seismic fault segmentation [ 5 ] , and cross-well prediction [ 20 ] . Hyperspectral and geophysical time-series analysis have similarly benefited from foundation-style encoders [ 18 , 21 ] . These advances collectively signal a paradigm shift from task-specific models toward domain-informed foundation models in Earth sciences. Meanwhile, foundation models have revolutionized natural language processing and computer vision by leveraging large-scale pretraining and token-based representation learning. Large language models (LLMs) employ subword tokenization (e.g., byte pair encoding, SentencePiece), while vision and seismic foundation models tokenize images or traces into patch embeddings or vector-quantized (VQ) tokens. These designs have proven effective for scaling Transformer architectures, stabilizing masked modeling objectives, and enabling cross-modal alignment. In geophysics, Chen et al. [ 8 ] proposed Gaia , a pretrained model for logging-curve reconstruction. Other studies explored generic time-series foundation models such as TOTEM [ 27 ] , TimeGPT [ 16 , 11 ] , and conformal prediction frameworks [ 1 ] . Broader reviews also highlight both the opportunities and challenges of subsurface foundation models [ 15 ] . However, most existing approaches either target single-curve reconstruction or remain domain-agnostic, leaving unresolved the compounded challenges of multi-curve heterogeneity, stratigraphy-aware representation, and multi-task geological interpretation. 1.3 This Work: WLFM Our key insight is to extend the tokenization paradigm to well-log interpretation. Instead of encoding continuous log patches directly as floating-point embeddings, we introduce a vector-quantized (VQ) tokenizer that discretizes multi-curve inputs into a symbolic space. This places WLFM within the same modeling framework as language and seismic foundation models, where tokens serve as the fundamental unit of representation. The unified token space offers three principal advantages: (i) a natural interface for multi-modal alignment, enabling well logs, seismic, and geological text to be jointly modeled in a shared embedding space; (ii) a structured and interpretable “geological vocabulary,” where tokens capture recurring morphologies such as shale–sand transitions or high-resistivity, low-porosity intervals; and (iii) improved robustness and generalization, as discrete tokens suppress amplitude shifts and noise while providing stable targets for masked modeling. Building on this principle, we propose WLFM, one of the first foundation models explicitly designed for multi-curve well-log interpretation. WLFM is pretrained from scratch on large-scale unlabeled datasets using a modular framework that combines tokenization, self-supervised representation learning, and multi-task fine-tuning. It incorporates domain-informed priors—relative-depth encodings to preserve stratigraphic context and curve-type embeddings to represent geophysical modalities—and employs a hardware-aware pretraining pipeline to ensure efficient large-scale training. 1.4 Contributions The contributions of this work are threefold: 1. We propose WLFM, a domain-specialized foundation model for multi-curve well-log interpretation, introducing a VQ tokenizer that transforms raw signals into a reusable “geological vocabulary” of discrete tokens. 2. We design a structurally grounded pretraining framework that integrates stratigraphy-aware contrastive learning with domain-informed priors and a hardware-aware patch loading strategy for efficient training at scale. 3. We demonstrate that WLFM, following a pretrain–then–finetune paradigm, consistently outperforms supervised and self-supervised baselines across lithology, porosity, reconstruction, and boundary tasks—particularly under cross-well, noisy, and low-label regimes. Furthermore, WLFM produces interpretable stratigraphic representations and achieves improved training efficiency. 2 Methods Figure 2: Detailed architecture of WLFM. The workflow comprises three stages: (1) Tokenization —depth-aligned log patches are discretized into geological tokens via vector quantization; (2) Self-supervised pretraining —a transformer backbone is optimized with masked-token modeling and stratigraphy-aware contrastive learning to capture both intra-log structure and cross-well semantics; (3) Task adaptation —lightweight heads are fine-tuned for lithology classification, porosity regression, and curve reconstruction. We propose WLFM, a modular three-stage framework for well-log representation learning (Fig. 2 ). Stage 1 converts continuous multi-curve logs into discrete tokens via a domain-aware tokenizer. Stage 2 pretrains a transformer backbone using masked-token modeling and stratigraphy-aware contrastive learning. Stage 3 enables downstream adaptation via multi-task fine-tuning. This design allows WLFM to learn from heterogeneous formations and generalize under limited supervision. 2.1 Domain-Aware Encoder for Multi-Curve Well Logs Well logging records consist of heterogeneous physical measurements acquired at uniform depth intervals.

Read the complete text