Generating Hierarchical JSON Representations of Scientific Sentences Using LLMs

Satya Sri Rajiteswari Nimmagadda, Ethan Young, Niladri Sengupta, Ananya Jana, Aniruddha Maiti · 2026

Read Generating Hierarchical JSON Representations of Scientific Sentences Using LLMs by Satya Sri Rajiteswari Nimmagadda, Ethan Young, Niladri Sengupta, Ananya Jana, Aniruddha Maiti (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23532.txt.

Introduction

Nimmagadda and four co-authors test whether tree-shaped JSON can hold the meaning of science sentences. They fine-tune a small language model with a new structural loss. The model turns each sentence into a layered JSON tree. A second model then rebuilds the sentence from that tree alone. The team compare the rebuild with the source by meaning and wording overlap. The match stays strong, which shows the format keeps the facts well. Such trees could help machines store and search science text.

Catalog record

CREATORSatya Sri Rajiteswari Nimmagadda, Ethan Young, Niladri Sengupta, Ananya Jana, Aniruddha Maiti
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE12244
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.23532

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generating Hierarchical JSON Representations of Scientific Sentences Using LLMs Satya Sri Rajiteswari Nimmagadda, Ethan Young, Niladri Sengupta, Ananya Jana, Aniruddha Maiti Abstract This paper investigates whether structured representations can preserve the meaning of scientific sentences. To test this, a lightweight LLM is fine-tuned using a novel structural loss function to generate hierarchical JSON structures from sentences collected from scientific articles. These JSONs are then used by a generative model to reconstruct the original text. Comparing the original and reconstructed sentences using semantic and lexical similarity we show that hierarchical formats are capable of retaining information of scientific texts effectively. I Introduction Scientific sentences often express a core claim along with additional information such as conditions, temporal qualifiers, or logical exceptions. Any representation used for processing scientific text must reflect this complex structure to avoid information loss. Previous methods mapped sentences to a single relation between two entities using a flat schema such as A–relation–B [ 1 ] . While suitable for simple assertions, this format cannot represent clause-level dependencies or hierarchical logic. This study uses a hierarchical JSON format that separates the core statement from supporting components. A lightweight language model is fine-tuned to generate such structures from raw sentences. The structured JSON output is then used by a generative model to reconstruct the original sentence. Reconstruction quality is measured using semantic and lexical similarity to assess whether the structured format generated by LLM preserved meaning. The method is evaluated on a diverse set of scientific sentences sampled from multiple scientific domains. II Background and Related Work Several studies examined sentence reconstruction from latent spaces [ 2 , 3 ] . Maiti et al. represented scientific sentences to a flat A–relation–B structure [ 1 ] . Wang and Bai [ 4 ] represented long sentences as dependency graphs and used graph network to capture grammatical and semantic relations. Studies on structured representation learning include work by Liu et al. [ 5 ] , Zheng et al. [ 6 ] . Das et al. [ 7 ] and Thilagam et al. [ 8 ] used JSON structures in their work. Yang et al. [ 9 ] introduced a benchmark for structured outputs in formats such as JSON, HTML, and YAML. Recent studies examined structured output from LLMs. Shorten et al. [ 10 ] reported variation in JSON formatting behavior in LLM responses. Dunn et al. [ 11 ] fine tuned GPT 3 for structured extraction from scientific text. Dagdelen et al. [ 12 ] extended this type of extraction to scientific records in JSON format. III Methodology III-A Dataset We constructed a dataset of 1500 scientific sentences taken from articles in seven scientific domains: physics, computer science, mathematics, economics, biology, chemistry, and medicine. Articles were chosen from arXiv, bioRxiv, ChemRxiv, and PubMed. Sentences with equations, symbols, or citation markers were excluded in the preprocessing step. After preprocessing, out of 1500 initial sentences, 1370 having complete meaning were retained. Table I reports the number of articles and sentences collected from each source. No more than six sentences were taken from one article. Domain Repository Articles Sentences Physics arXiv 97 447 Computer Science arXiv 39 193 Mathematics arXiv 25 113 Economics arXiv 50 249 Biology bioRxiv 51 256 Chemistry ChemRxiv 12 58 Medicine PubMed 17 54 Total 4 (unique) 291 1370 TABLE I: Dataset summary by domain and repository. For experiments, we divided the collected and cleaned data into 1096 training and validation (n=138) sentences, and 274 test sentences. III-B Generation of Training Data GPT-4o is used to generate high-quality training data. The prompt contained instruction to generate output JSONs from input sentences containing two main parts: core: This part has two keys: a generic label, and a central claim of the sentence so that the most important information of the sentence can be retained. hierarchy: This part contained the hierarchical structure around the core. Each level of the hierarchy contains two important keys: a relationship type and components connected by that relationship type. The generative model is instructed to choose relation types freely, but 17 predefined relationship types were given as a reference or example. These relationship types are a refined list used in prior work [ 1 ] . The JSON generating prompt contained instructions to: (a) avoid word-to-word copying from the input; (b) compress the contents so that no single field (either in core or any hierarchy) exceeds 30% of the original sentence length; (c) organize content hierarchically as per relationship types based on logical flow of the sentence. In the prompt, we also provided few shot examples showing sentences and their ideal JSON representations. III-C Fine Tuning of LLM We selected Mistral-7B model [ 13 ] , to demonstrate and validate the idea. The selected model is a representative of \sim 7B parameter LLM class. The model is fine-tuned to predict the structured output sequence from the input sentences using low-rank adaptation (LoRA) technique. Algorithm 1 Novel Structural Loss Function used in this study 0: Generated token logits from causal language model 1: Compute base loss: \mathcal{L}_{\text{CE}}\leftarrow CrossEntropyLoss 2: Decode predictions: \hat{y}\leftarrow\text{ArgMax}(\text{logits}) 3: Initialize JSON failure count: f\leftarrow 0 4: for each decoded string s in \hat{y} do 5: # Attempt to extract structured output: 6: j\leftarrow\text{JSONParser}(s) 7: if JSON parsing fails then 8: f\leftarrow f+1 9: end if 10: end for 11: Compute structure penalty: \mathcal{L}_{\text{struct}}\leftarrow f/|\hat{y}| 12: Combine losses: \mathcal{L}_{\text{total}}\leftarrow\mathcal{L}_{\text{CE}}+\mathcal{L}_{\text{struct}} 13: return \mathcal{L}_{\text{total}} To enforce syntactic correctness in structured outputs, we used a novel training mechanism that integrates a runtime structure validation agent into the training loop. Along with token-level losses, the proposed method also used penalty by dynamically evaluating whether the generated output is a valid JSON object. This is achieved by decoding each generated sample and applying an external parser (along with a Python try–except block) to check JSON validity. If parsing fails, a penalty is applied to the batch loss. We described this process in Algorithm 1 . The trained model was then used to generate structured JSON representations of the held-out test set. III-D Sentence Reconstruction To assess whether JSONs generated from the test data by Mistral-7B model retained sufficient information to reconstruct the original scientific sentences, they were used as input to a generative model (GPT-4o) API. We created a prompt that instructs the API to produce a natural language reconstruction of the given JSON. The generative model was tasked to interpret the core claim, hierarchical components and relation types and generated one coherent sentence per input. IV Experiments IV-A Training Configuration and Model Details We fine-tuned the Mistral-7B-v0.1 model using the training and validation data. LoRA adapters were applied to attention and feedforward projection layers, the rest of the model was kept frozen. The LoRA [ 14 ] configuration used rank 16, scaling factor 16, and dropout of 0.05. Training was performed using fully sharded data parallelism with automatic layer wrapping. We enabled gradient checkpointing to reduce memory use. We also used bfloat16 precision to reduce memory footprints. The model was trained for five epochs. The per-device batch size was 1, with gradient accumulation over 4 steps. The learning rate was set to 2\times 10^{-4} with a cosine decay schedule and a warmup ratio of 0.1. The maximum gradient norm was

Read the complete text