Contextual image caption creation using object positional embedding and generative models

Muhammad Danyal, Muhammad Roman, Abdul Shahid, Muhammad Yahya · 2026

Read Contextual image caption creation using object positional embedding and generative models by Muhammad Danyal, Muhammad Roman, Abdul Shahid, Muhammad Yahya (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/plos-pone-0353466.txt.

Description

Muhammad Danyal and three colleagues present this 2026 PLOS One paper on teaching computers to write captions that describe what is happening in a picture. Many automatic caption systems match words without grasping how objects relate to each other, so their captions come out thin. The team combined an object-detection model with a text-generating transformer that also encodes where objects sit in the image.

Against two older approaches, the new system scored best on measures that reward meaning and object relationships, even though it trailed on one word-matching score. Human experts also rated its captions highly for accuracy across several hundred test images. The work points toward image descriptions that are more useful for search, accessibility, and content sorting.

Catalog record

CREATORMuhammad Danyal, Muhammad Roman, Abdul Shahid, Muhammad Yahya
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41787
LICENSECC_BY
RIGHTSCC BY
SOURCEPLOS

Source: https://doi.org/10.1371/journal.pone.0353466

Browse the full library · About the Tower of Babel