Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini

Madhuri Shanbhogue, Zhe Li, Shanfeng Zhang, Gustavo Hernández Ábrego, Shih-Cheng Huang, Aashi Jain, Daniel Salz, Sonam Goenka, Chaitra Hegde, Ji Ma, Feiyang Chen, Jiaxing Wu, Tanmaya Dabral, Babak Samari, Kevin Poulet, Daniel Cer, Kaifeng Chen, Paul Suganathan, Hui Hui, Jovan Andonov, Philippe Schlattner, Jay Han, Iftekhar Naim, Wing Lowe, Vladimir Pchelin, Albert Yang, Yi-Ting Chen, Zhongli Ding, Grace Zhang, Georg Heigold, Yichang Chen, Antoine Reveillon, Brendan Mccloskey, Wenlei Zhou, Dahun Kim, Rui Meng, Emma Wang, Jack Zheng, Halley Fede, Zhen Yang, Keegan Mosley, Brian Potetz, Sahil Dua, Henrique Schechter Vera, Shen Gao, Hesen Zhang, Andreas Hess, Hengxuan Ying, Alberto Montes, Karan Gill, Min Choi, Sebastian Russo, Anja Hauth, Jinhyuk Lee, Michael Boratko, Megan Barnes, Vikram Rao, Claudiu Musat, Cyril Allauzen, Ehsan Variani, Shankar Kumar, Tom Bagby, Junyi Jiao, Yang Gu, Tengxin Li, Ayush Agrawal, Roberto Santana, Dev Nath, Stephen Karukas, Shuoxuan Han, Lucia Loher, Alice Twu, Nidhi Vyas, Siddharth Bhai, Frank Palma Gomez, Wangyuan Zhang, Chaoren Liu, Jizheng Yang, Steve Qiu, Shijie Zhang, Sujay Kulkarni, Sascha Rothe, Sean Nakamoto, Raphael Hoffmann, Zach Gleicher, Yunhsuan Sung, Qin Yin, Tom Duerig, Mojtaba Seyedhosseini · 2026

Read Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini by Madhuri Shanbhogue, Zhe Li, Shanfeng Zhang, Gustavo Hernández Ábrego, Shih-Cheng Huang, Aashi Jain, Daniel Salz, Sonam Goenka, Chaitra Hegde, Ji Ma, Feiyang Chen, Jiaxing Wu, Tanmaya Dabral, Babak Samari, Kevin Poulet, Daniel Cer, Kaifeng Chen, Paul Suganathan, Hui Hui, Jovan Andonov, Philippe Schlattner, Jay Han, Iftekhar Naim, Wing Lowe, Vladimir Pchelin, Albert Yang, Yi-Ting Chen, Zhongli Ding, Grace Zhang, Georg Heigold, Yichang Chen, Antoine Reveillon, Brendan Mccloskey, Wenlei Zhou, Dahun Kim, Rui Meng, Emma Wang, Jack Zheng, Halley Fede, Zhen Yang, Keegan Mosley, Brian Potetz, Sahil Dua, Henrique Schechter Vera, Shen Gao, Hesen Zhang, Andreas Hess, Hengxuan Ying, Alberto Montes, Karan Gill, Min Choi, Sebastian Russo, Anja Hauth, Jinhyuk Lee, Michael Boratko, Megan Barnes, Vikram Rao, Claudiu Musat, Cyril Allauzen, Ehsan Variani, Shankar Kumar, Tom Bagby, Junyi Jiao, Yang Gu, Tengxin Li, Ayush Agrawal, Roberto Santana, Dev Nath, Stephen Karukas, Shuoxuan Han, Lucia Loher, Alice Twu, Nidhi Vyas, Siddharth Bhai, Frank Palma Gomez, Wangyuan Zhang, Chaoren Liu, Jizheng Yang, Steve Qiu, Shijie Zhang, Sujay Kulkarni, Sascha Rothe, Sean Nakamoto, Raphael Hoffmann, Zach Gleicher, Yunhsuan Sung, Qin Yin, Tom Duerig, Mojtaba Seyedhosseini (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27295.txt.

Introduction

Madhuri Shanbhogue and colleagues introduce Gemini Embedding 2, a single model that embeds video, audio, image, and text together. Trained with large-scale contrastive learning across stages and tasks, it leads embedding benchmarks for retrieval in many forms. Its zero-shot range spans fields as different as astronomy, bioscience, fine arts, and cooking, suiting search and recommendation uses.

Catalog record

CREATORMadhuri Shanbhogue, Zhe Li, Shanfeng Zhang, Gustavo Hernández Ábrego, Shih-Cheng Huang, Aashi Jain, Daniel Salz, Sonam Goenka, Chaitra Hegde, Ji Ma, Feiyang Chen, Jiaxing Wu, Tanmaya Dabral, Babak Samari, Kevin Poulet, Daniel Cer, Kaifeng Chen, Paul Suganathan, Hui Hui, Jovan Andonov, Philippe Schlattner, Jay Han, Iftekhar Naim, Wing Lowe, Vladimir Pchelin, Albert Yang, Yi-Ting Chen, Zhongli Ding, Grace Zhang, Georg Heigold, Yichang Chen, Antoine Reveillon, Brendan Mccloskey, Wenlei Zhou, Dahun Kim, Rui Meng, Emma Wang, Jack Zheng, Halley Fede, Zhen Yang, Keegan Mosley, Brian Potetz, Sahil Dua, Henrique Schechter Vera, Shen Gao, Hesen Zhang, Andreas Hess, Hengxuan Ying, Alberto Montes, Karan Gill, Min Choi, Sebastian Russo, Anja Hauth, Jinhyuk Lee, Michael Boratko, Megan Barnes, Vikram Rao, Claudiu Musat, Cyril Allauzen, Ehsan Variani, Shankar Kumar, Tom Bagby, Junyi Jiao, Yang Gu, Tengxin Li, Ayush Agrawal, Roberto Santana, Dev Nath, Stephen Karukas, Shuoxuan Han, Lucia Loher, Alice Twu, Nidhi Vyas, Siddharth Bhai, Frank Palma Gomez, Wangyuan Zhang, Chaoren Liu, Jizheng Yang, Steve Qiu, Shijie Zhang, Sujay Kulkarni, Sascha Rothe, Sean Nakamoto, Raphael Hoffmann, Zach Gleicher, Yunhsuan Sung, Qin Yin, Tom Duerig, Mojtaba Seyedhosseini
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45635
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27295

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini See Contributions and Acknowledgments section. ∗ Equal contributions. Madhuri Shanbhogue, Zhe Li, Shanfeng Zhang, Gustavo Hernández Ábrego, Shih-Cheng Huang, Aashi Jain, Daniel Salz, Sonam Goenka, Chaitra Hegde, Ji Ma, Feiyang Chen, Jiaxing Wu, Tanmaya Dabral, Babak Samari, Kevin Poulet, Daniel Cer, Kaifeng Chen, Paul Suganathan, Hui Hui, Jovan Andonov, Philippe Schlattner, Jay Han, Iftekhar Naim, Wing Lowe, Vladimir Pchelin, Albert Yang, Yi-Ting Chen, Zhongli Ding, Grace Zhang, Georg Heigold, Yichang Chen, Antoine Reveillon, Brendan Mccloskey, Wenlei Zhou, Dahun Kim, Rui Meng, Emma Wang, Jack Zheng, Halley Fede, Zhen Yang, Keegan Mosley, Brian Potetz, Sahil Dua, Henrique Schechter Vera, Shen Gao, Hesen Zhang, Andreas Hess, Hengxuan Ying, Alberto Montes, Karan Gill, Min Choi, Sebastian Russo, Anja Hauth, Jinhyuk Lee, Michael Boratko, Megan Barnes, Vikram Rao, Claudiu Musat, Cyril Allauzen, Ehsan Variani, Shankar Kumar, Tom Bagby, Junyi Jiao, Yang Gu, Tengxin Li, Ayush Agrawal, Roberto Santana, Dev Nath, Stephen Karukas, Shuoxuan Han, Lucia Loher, Alice Twu, Nidhi Vyas, Siddharth Bhai, Frank Palma Gomez, Wangyuan Zhang, Chaoren Liu, Jizheng Yang, Steve Qiu, Shijie Zhang, Sujay Kulkarni, Sascha Rothe, Sean Nakamoto, Raphael Hoffmann, Zach Gleicher, Yunhsuan Sung, Qin Yin, Tom Duerig, Mojtaba Seyedhosseini Abstract We introduce Gemini Embedding 2, a native multimodal embedding model that allows embedding video, audio, image, and text modalities in a unified representation space. We leverage the multimodal capabilities of Gemini to produce embeddings for arbitrary combinations of interleaved inputs across all these modalities that generalize well across a wide variety of tasks. Applying large-scale contrastive learning in a multi-task multi-stage training setup, we achieve state-of-the-art performance on key embedding benchmarks including unimodal, cross-modal, and multimodal retrieval spanning a diverse set of tasks. We show that our embedding model demonstrates strong performance (with a score of 62.9 R@1 on MSCOCO, 68.8 NDCG@10 on Vatex, 69.9 on MTEB multilingual and 84.0 on MTEB Code) across a variety of tasks surpassing the performance of specialized models. These unified capabilities make Gemini Embedding 2 a promising candidate for downstream use cases such as RAG, recommendation and search. Furthermore, its robust zero-shot performance across distinct fields – from astronomy and bioscience to fine arts and the culinary arts – establishes it as a highly reliable, out-of-the-box representation even for specialized domains. 1 Introduction Embedding models provide dense vector representations capturing semantic information that is crucial for adaptation in a wide range of downstream tasks. With foundational models being natively multimodal and powered with exceptionally growing capabilities, it is important to ensure embedding models capture semantic information within and across all modalities in a coherent manner. Such general-purpose embedding models will also enhance the performance across a broad spectrum of applications like video recommendations and document search which are rich in information across different modalities but since the contained modalities are not inherently homogenous, they can benefit from having rich semantic information from across all modalities. Existing multimodal embedding models like CLIP [ 34 ] , ALIGN [ 14 ] , SigLIP 2 [ 37 ] , CoCa [ 44 ] embed heterogenous modalities by using paired cross-modal data and training modality-specific encoders to encode them into a unified vector space. This late-fusion approach results in good unimodal and cross-modal capabilities but has a key limitation in handling mixed-modality inputs and lacks richness since it does not utilize interactions between modalities. With advances in Multimodal Large Language Models (MLLMs), it is now possible to achieve semantically richer embeddings enabled by the deep fusion of cross-modal interactions. Figure 1: Conceptual overview of the Gemini Embedding 2 workflow. The model natively processes heterogeneous inputs—text, images, video, audio, documents, and their combinations—mapping them into a single, unified high-dimensional vector space where cross-modal semantic relationships are preserved. In this work, we introduce a generalizable multimodal embedding model that embeds video, audio, image, text modalities, and any arbitrary combination thereof into a single representation space. The multimodal Gemini Embedding 2 is trained by leveraging Gemini’s [ 6 ] capabilities and utilizing multi-task training with a diverse set of tasks resulting in a model that captures various interactions between modalities. Figure 1 shows a high-level representation of how multimodal Gemini Embedding 2 maps the heterogenous sources into a unified vector space. The curated set of tasks help the model generalize across a wide variety of enterprise use cases like document retrieval, video recommendation, audio-based search, and RAG applications [ 19 ] . Crucially, enabling the model to handle interleaved sequences of images, text, and video facilitates complex, novel retrieval paradigms—such as zeroing in on specific temporal events in a video using combined visual and textual prompts. Using Gemini’s capabilities we also show that native audio understanding and native multimodal understanding outperforms text-based alternatives like ASR or captioning. Figure 2: Gemini Embedding 2 shows strong performance across multimodal retrieval tasks spanning image, text, video, and document modalities. ∗ MTEB number is reported for Voyage-3.5 since Voyage-3.5-multimodal does not report MTEB . We evaluate comprehensively on a wide variety of benchmarks, both academic-focused and enterprise-focused. As shown in Figure 2 , our model achieves state-of-the-art performance compared to other models. For evaluating the text embedding capabilities, we rely on the Massive Multilingual Text Embedding Benchmark (MMTEB) [ 10 ] which consists of multi-lingual tasks spanning key downstream embedding use cases like retrieval, clustering, classification, etc. Gemini Embedding 2 achieves state-of-the-art performance on multilingual and code surpassing existing models on the leaderboard. We demonstrate strong numbers on a broad range of cross-modal retrieval benchmarks like MSCOCO [ 5 ] , Flickr30k [ 32 ] , and MSR-VTT [ 42 ] . We also demonstrate the model’s ability to generalize to most multimodal retrieval tasks in general as well as specialized domains. 2 Related Work Large Language Models as Text Embedders The paradigm of text embedding models has matured from relying on purely encoder-only architectures (e.g., BERT [ 9 ] , RoBERTa [ 23 ] ) to utilizing decoder-only or massive LLM backbones. Models such as the BGE [ 4 ] series and E5 [ 39 ] established instruction-tuned representations, effectively unifying downstream tasks—like semantic search, clustering, and classification—into a single model via task-specific prefixes. Recognizing the rich semantic understanding capabilities of LLMs, recent research has focused heavily on LLM-augmented training and distillation. The Gecko model [ 18 ] demonstrated that lightweight, highly-efficient retrievers can be trained through a two-step distillation pipeline that leverages the vast knowledge of massive LLM teachers. Concurrently, NV-Embed [ 16 ] achieved strong performance on the MMTEB leaderboard [ 29 ] by transforming decoder-only LLMs into generalist embedders using instruction-tuned contrastive learning and the aggressive integration of synthetic, non-retrieval data. Gemini Embedding [ 17 ] demonstrated state-of-the-art performance on the MMTEB leaderboard due to utilizing synthetic data and excellent generalization to multilingual tasks through the powerful pre-training of Gemini. Evolution of Multimodal Embedders Early multimodal embedding paradigms, exemplified by dual-tower models like CLIP [ 34 ] and ALIGN [ 14 ] , were limited by their reliance on narrow contrastive learning objectives over simple image–text pairs. Today, the field is gravitating towards multimodal architectures capable of mapping text, code, images, structured documents, audio, and video into a single, unified, continuous semantic space. Embedding models are trained by extending existing MLLMs

Read the complete text