Batch Augmentation with Unimodal Fine-tuning for Multimodal Fusion of Large Language Models
H M Dipu Kabir, Subrota Kumar Mondal, Mohammad Ali Moni · 2025
Read Batch Augmentation with Unimodal Fine-tuning for Multimodal Fusion of Large Language Models by H M Dipu Kabir, Subrota Kumar Mondal, Mohammad Ali Moni (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06592.txt.
Description
H M Dipu Kabir and co-authors propose a 2025 training method for multimodal large language models. The method starts from pretrained models that each handle one kind of input. Each model is first fine-tuned alone, then joined through a Multi-Layer Perceptron head.
Training then uses batch augmentation across the combined model and its parts. Some parts can be swapped for scripts or AI agents when the task allows it. Tests used the FPU23 ultrasound set and the UPMC Food-101 set. The combined model gave the best average result among the methods tried. A shared dataloader script is also provided so others can repeat the training steps.
Catalog record
| CREATOR | H M Dipu Kabir, Subrota Kumar Mondal, Mohammad Ali Moni |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-06592