Distributed Zero-Shot Learning for Visual Recognition

Zhi Chen, Yadan Luo, Zi Huang, Jingjing Li, Sen Wang, Xin Yu · 2025

Read Distributed Zero-Shot Learning for Visual Recognition by Zhi Chen, Yadan Luo, Zi Huang, Jingjing Li, Sen Wang, Xin Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08170.txt.

Introduction

Zhi Chen, Yadan Luo, Zi Huang, and Jingjing Li propose distributed zero shot learning for recognizing classes never seen during training, using data spread across separate nodes. Zero shot methods link images to class attributes so new categories can be identified by description alone. Data held at different sites varies in character, which can destabilize the shared attribute space.

A cross node regularizer keeps distances between attribute features similar at every site. A global consensus rule then forces the mapping between attributes and visual features to agree in both directions across nodes. Together these pieces reduce biased mappings from single sites, and experiments show clear gains over leading distributed approaches.

Catalog record

CREATORZhi Chen, Yadan Luo, Zi Huang, Jingjing Li, Sen Wang, Xin Yu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-08170

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Distributed Zero-Shot Learning for Visual Recognition Zhi Chen, Yadan Luo, Zi Huang, Jingjing Li, Sen Wang, Xin Yu Abstract In this paper, we propose a Distributed Zero-Shot Learning (DistZSL) framework that can fully exploit decentralized data to learn an effective model for unseen classes. Considering the data heterogeneity issues across distributed nodes, we introduce two key components to ensure the effective learning of DistZSL: a cross-node attribute regularizer and a global attribute-to-visual consensus. Our proposed cross-node attribute regularizer enforces the distances between attribute features to be similar across different nodes. In this manner, the overall attribute feature space would be stable during learning, and thus facilitate the establishment of visual-to-attribute (V2A) relationships. Then, we introduce the global attribute-to-visual consensus to mitigate biased V2A mappings learned from individual nodes. Specifically, we enforce the bilateral mapping between the attribute and visual feature distributions to be consistent across different nodes. Thus, the learned consistent V2A mapping can significantly enhance zero-shot learning across different nodes. Extensive experiments demonstrate that DistZSL achieves superior performance to the state-of-the-art in learning from distributed data. I Introduction Visual recognition aims to identify and categorize visual data, forming a cornerstone in the field of computer vision. With the ever-growing amount of data, the ability to recognize instances from previously seen and unseen classes is highly desired. To this end, Generalized Zero-Shot Learning (GZSL) [ 73 , 47 , 62 , 59 , 26 ] has been provided. GZSL approaches usually first establish a visual-attribute mapping and then exploit it to classify seen classes ( i.e. , available in both training and test) and unseen classes ( i.e. , only appear in testing) based on their attribute descriptions. Current GZSL methods often require a large amount of centralized data in training. However, when data cannot be shared or centralized, previous methods might fail to achieve satisfactory performance. Fig. 1: An illustration of Distributed Zero-Shot Learning (DistZSL), which aims to infuse ZSL capability into distributed learning frameworks. Federated Learning (FL) is considered an appealing distributed learning framework, as it only exchanges model parameters instead of original data. Direct incorporation of existing GZSL methods into FL may not lead to adequate performance. This is because models trained on individual nodes would yield different visual-attribute mappings, and the inconsistent mappings do not help zero-shot classification. These mappings will become even more inconsistent when the data are distributed heterogeneously. In this paper, we present a Distributed Zero-Shot Learning (DistZSL) framework that can learn a GZSL model from multiple decentralized data sources. Here, we assume each node does not have overlapping classes and we denote this distribution as a partial class-conditional distribution ( p.c.c.d. ). This assumption imposes more challenges for DistZSL: (1) different models would struggle to learn a consistent visual-to-attribute (V2A) mapping as each node learns V2A independently; (2) trained models on different nodes would also bias to local data; For instance, a model trained solely on birds with gray and brown wings may struggle to differentiate between blue and black wings in testing. To tackle the aforementioned challenges, we introduce two key components to our DistZSL. To be specific, we design a cross-node attribute regularizer to stabilize the distribution of attributes across nodes. Then, we present a global attribute-to-visual consensus to mitigate inconsistency among V2A mappings learned at different nodes. Note that both the cross-node attribute regularizer and the global attribute-to-visual consensus are applied on the client side. Our cross-node attribute regularizer is designed to enforce the distances between attribute features to be similar across different nodes. First, to estimate the inter-class attribute distances, the central node constructs a sparse similarity matrix using Graphical Lasso [ 23 ] . Then, this similarity matrix is shared among individual nodes, acting as a cross-node reference during distributed training. Moreover, in local training, we employ the KL divergence to measure and minimize the distance between the predicted class-wise similarities and the constructed similarity matrix. Our global attribute-to-visual consensus is introduced to mitigate the biased V2A mappings learned from different nodes. Since the attribute regularizer has stabilized the attribute feature distribution, a consistent V2A mapping can be achieved by learning an attribute-to-visual is achieved by establishing a bilateral connection between semantic and visual features. This strategy can improve the accuracy of attribute prediction and further mitigate local bias. In addition to predicting attributes from visual features, the bilateral connection reconstructs the visual features from the predicted attributes. We apply a bilateral loss on the differences between the reconstructed visual features and the original ones. By minimizing the bilateral loss, the reconstruction forces the predicted attributes to accurately maintain visual information. Thus, we can enhance the accuracy of the predicted attributes. Fig. 2: Attribute-based learning allows local models to learn towards the global minima across devices. In contrast, attribute-free learning simply averages the classifier weights of individual clients, leading to local optima. To evaluate our proposed DistZSL, we incorporate four state-of-the-art ZSL methods into six representative FL frameworks, resulting in 24 baseline models. Extensive experiments on three benchmark ZSL datasets demonstrate that our DistZSL consistently outperforms the baseline models. In addition, we also evaluate DistZSL under different scenarios of distributed data, such as handling insufficient samples ( e.g. , 6 samples per class), fine-grained classes ( e.g. , birds), and an extensive number of classes, as well as different data distributions ( i.i.d. , non-i.i.d. and p.c.c.d. ). Comprehensive ablation studies also demonstrate the effectiveness of our proposed components in DistZSL. To summarize, the main contributions of this work are listed as follows: • We propose DistZSL, which infuses the ZSL ability into FL frameworks. We identify that attribute-based learning in ZSL can inherently benefit decentralized training. • We pinpoint two critical challenges: decentralized data, and biased local updates. To address these issues, our proposed solution integrates a cross-device attribute regularizer and a bilateral semantic-visual connection. • Through comprehensive experiments of our proposed method and various baselines on three ZSL datasets, we demonstrate the capability of addressing the identified challenges in various settings. II Related Work II-A Zero-Shot Learning Zero-Shot Learning (ZSL) [ 25 , 78 , 76 , 85 , 18 , 10 , 27 , 13 ] addresses a challenging problem in computer vision [ 80 , 7 , 68 , 17 , 83 , 44 , 87 , 70 , 15 , 69 , 67 , 79 , 71 , 86 , 16 , 88 , 20 ] where the test set contains additional classes not presented during training. To bridge the seen and unseen classes [ 1 , 45 , 77 ] , a standard solution is learning the visual-semantic relationships. Intermediate class-level semantic representations include attribute annotations [ 37 ] , natural language descriptions [ 21 ] , etc. In general, to learn the visual-semantic relationships, there are two streams of methods: embedding-based methods [ 75 , 57 , 56 ] and generative methods [ 73 , 89 , 28 ] . The former group projects the visual and semantic information to the same feature space. The learned projection can then infer the class attributes for the samples of unseen classes. Various methods have been proposed in this research direction, including graph learning [ 46 ] , attentive learning [ 48 ,

Read the complete text