Self-Supervised Learning of Synapse Types from EM Images
Aarav V Shetty, Gary B Huang · 2025
Read Self-Supervised Learning of Synapse Types from EM Images by Aarav V Shetty, Gary B Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21663.txt.
Introduction
Aarav Shetty and Gary B. Huang sort synapses into types without being shown any labeled examples, reading only electron microscope images of fly brains. Knowing a synapse's type can hint at its chemical messenger or at whether its strength can be tuned, yet hand-labeling enough examples to train a classifier is slow. The true number of types is unknown beforehand.
Their unsupervised trick leans on biology: synapses sitting near each other on the same neuron probably resemble each other more than random pairs drawn across cells. Learning from that neighborhood signal alone, the method separates synapses into classes whose count it discovers rather than assumes. Beyond the immediate sorting, the classes can help experts choose ground-truth examples that span the real range of synapse shapes for future studies.
Catalog record
| CREATOR | Aarav V Shetty, Gary B Huang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 10 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21663
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Self-Supervised Learning of Synapse Types from EM Images Aarav V Shetty, Gary B Huang Abstract Separating synapses into different classes based on their appearance in EM images has many applications in biology. Examples may include assigning a neurotransmitter to a particular class, or separating synapses whose strength can be modulated from those whose strength is fixed. Traditionally, this has been done in a supervised manner, giving the classification algorithm examples of the different classes. Here we instead separate synapses into classes based only on the observation that nearby synapses in the same neuron are likely more similar than synapses chosen randomly from different cells. We apply our methodology to data from Drosophila . Our approach has the advantage that the number of synapse types does not need to be known in advance. It may also provide a principled way to select ground-truth that spans the range of synapse structure. 1 Introduction Separating synapses into different classes based on their appearance in EM images has many applications in biology. Examples include assigning a neurotransmitter to a particular class or separating synapses whose strength can be modulated from those whose strength is fixed. Initial machine learning techniques concentrated on detecting synapses, not classifying them. Even this is not simple, so it was somewhat of a surprise that synapses could not only be identified, but also divided into classes based on their putative neurotransmitter [ Eckstein et al. 2024 ] . This classification is not obvious to humans from the image data, and significant work has gone into trying to understand what features the machine learning is using to make this distinction [ Adjavon et al. 2024 ] . Traditionally, both these tasks have been done in a supervised manner, giving the classification algorithm examples of the different classes: synapses/not synapses, or examples with known neurotransmitters. Supervised learning requires labeled ground-truth, meaning that each training example must have an associated label describing its class. In contrast, self-supervised learning eliminates the need for explicit labels by using the data itself to generate a learning signal. Contrastive methods used in our research, such as the NT-Xent loss function, train models to bring representations of similar inputs closer together while pushing apart dissimilar ones [ Chen et al. 2020 ] . Here, we instead separate synapses into classes based only on the observation that nearby synapses in the same neuron are likely more similar than synapses chosen randomly from different cells. We apply this approach and analyze the resulting learned representations in Drosophila , a model system for which extensive ground-truth is known. Self-supervised classification of synapse types has not been previously attempted, but it potentially has several advantages and uses. The main advantage is that ground-truth is not required. Even the number of synapse types does not need to be known in advance. Another plus is that supervised methods seem quite sensitive to imaging conditions and the exact species used. This often means that ground-truth cannot be transferred even to closely related data - for example, ground-truth for one species of Drosophila , or even one particular dataset, cannot be used to classify data from another. Self-supervised classification may also provide a principled way to select ground-truth that spans the range of synapse structure. This range, illustrated through techniques such as UMAP, can show what fraction of useful structural differences are covered by the ground-truth. Selecting examples in portions of this space that are not well covered may be particularly helpful in improving classification among the supervised classifiers. This could be particularly useful when working with novel species, where ground-truth can be expensive to acquire (perhaps requiring biochemical and genetic methods). 2 Previous work The current state of the art in synapse classification is shown in Fig. 1 . This is the result of a supervised classification of synapse types, where the classification algorithm is given ground-truth consisting of synapses of known transmitter types. Figure 1 : Synapses classified by neurotransmitter type. Two dimensional UMAP reduction of classification output. Each neurotransmitter is given a distinct color. This ground-truth is difficult and expensive to obtain, requiring genetic and biochemical evidence for the transmitters of a particular cell type. In particular it requires at least a few cells of every transmitter type we hope to separate. This can be a serious limitation for transmitter types that are used only a few less numerous cells. 3 Methods Dales’s principle [ Dale 1935 ] , which is actually a heuristic, specifies that all the synapses of the same neuron use the same neurotransmitter(s). Furthermore, segmentation divides an EM volume into supervoxels , which are sets of voxels that segmentation is very sure belong to the same neuron. Therefore, from a classification viewpoint, we would expect all synapses from the same supervoxel to share the same classification. We rely on having a segmentation that is accurate and conservative, meaning it should contain as few false merges between different neurons as possible. A false merge occurs when parts of two distinct neurons are incorrectly joined into a single segment, leading to mixed neuronal identities and potentially misleading classifications. To achieve accurate segmentation, we use the Flood-Filling Networks method developed by Google, which is designed to generate precise segmentations with minimal false merges in large-scale EM datasets [ Januszewski et al. 2017 ] . Conversely, we would expect that comparing synapses from different supervoxels would result in a very different distribution. Some will be in the same class (those that share the same transmitter), but others will differ. The question then is whether self-supervised machine learning can learn this distinction. In order to examine each synapse, we used a Visual Geometry Group(VGG) model [ Simonyan and Zisserman 2014 ] . We used an NT_Xent loss function for the Self-Supervised model [ Chen et al. 2020 ] . The function essentially gives a low loss value when two neurotransmitters that are known to be the same are mapped to similar representations in the learned high dimensional space. The network had an input receptive field size of 80 3 voxels, of monochrome images taken for the purpose of EM connectomics, with an isotropic 8x8x8 nm resolution. One of the main advantages of this data is that existing neurotransmitter ID had already been run, providing a way to test our results. Each subvolume was centered on a synapse in the volume, as determined by a previous step of synapse detection. In addition, all synapses were labeled with the ID of the supervoxel that contained them. We trained this network on an EM dataset of the male brain of Drosophila , the ‘CNS’ dataset [ Berg et al. 2025 ] . ’CNS’ has an image volume of 160 teravoxels. The dataset contains 166,696 proofread brain and ventral nerve cord neurons. 4 Results We achieve a self-supervised separation as shown in Fig. 2 . To visualize the structure of the learned representations, we applied Uniform Manifold Approximation and Projection(UMAP) [ McInnes et al. 2020 ] . UMAP is a nonlinear dimensionality reduction technique that reduces high-dimensional data to a low-dimensional space while preserving important structure, making it easier to see patterns. In our case, UMAP was applied to the penultimate representations extracted from trained networks, providing a