Contextual image caption creation using object positional embedding and generative models
Muhammad Danyal, Muhammad Roman, Abdul Shahid, Muhammad Yahya · 2026
Read Contextual image caption creation using object positional embedding and generative models by Muhammad Danyal, Muhammad Roman, Abdul Shahid, Muhammad Yahya (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/plos-pone-0353466.txt.
Description
Muhammad Danyal and three colleagues present this 2026 PLOS One paper on teaching computers to write captions that describe what is happening in a picture. Many automatic caption systems match words without grasping how objects relate to each other, so their captions come out thin. The team combined an object-detection model with a text-generating transformer that also encodes where objects sit in the image.
Against two older approaches, the new system scored best on measures that reward meaning and object relationships, even though it trailed on one word-matching score. Human experts also rated its captions highly for accuracy across several hundred test images. The work points toward image descriptions that are more useful for search, accessibility, and content sorting.
Catalog record
| CREATOR | Muhammad Danyal, Muhammad Roman, Abdul Shahid, Muhammad Yahya |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41787 |
| LICENSE | CC_BY |
| RIGHTS | CC BY |
| SOURCE | PLOS |