Local-Order Auxiliary Losses Can Improve Autoencoder Reconstruction
Harvey Dam, Martin Burtscher, Tripti Agarwal, Ganesh Gopalakrishnan · 2025
Read Local-Order Auxiliary Losses Can Improve Autoencoder Reconstruction by Harvey Dam, Martin Burtscher, Tripti Agarwal, Ganesh Gopalakrishnan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-04202.txt.
Introduction
Harvey Dam and co-authors report a machine learning study, published in 2025, on training autoencoders to rebuild data more faithfully. Autoencoders squeeze data into a small latent code and then expand it again. Most training uses mean squared error, which checks each point alone and can miss whether nearby points keep their local order.
The team adds a new helper loss called finite-difference sign error, which punishes cases where the rebuilt data rises or falls in the wrong direction next to its neighbors. Across four test tasks, mixing this loss with the usual one cut error by a large margin. Gains were strongest for smooth spatial fields, while the helper loss alone did poorly.
Catalog record
| CREATOR | Harvey Dam, Martin Burtscher, Tripti Agarwal, Ganesh Gopalakrishnan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 106 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-04202
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Local-Order Auxiliary Losses Can Improve Autoencoder Reconstruction Harvey Dam, Martin Burtscher, Tripti Agarwal, Ganesh Gopalakrishnan Abstract Mean-squared error is the default objective for training autoencoders, yet compressed reconstructions often depend not only on pointwise accuracy but also on preserving local spatial order. We study whether structural auxiliary losses can improve, rather than trade off against, MSE in finite-capacity autoencoders. We introduce finite-difference sign error (FDSE), a local-order auxiliary objective that penalizes disagreements between the signs of neighboring finite differences in the target and reconstruction. FDSE is simple, architecture-agnostic, and differentiable through smooth sign surrogates. Across four tensor reconstruction tasks, we find that moderate mixtures of MSE and FDSE can substantially reduce validation MSE relative to pure MSE training. In coefficient sweeps, FDSE mixtures reduce validation MSE by 2.3 × \times –7.0 × \times over pure MSE on these tasks, while comparisons with other auxiliary objectives show FDSE to be among the strongest structural objectives tested. The effect is not universal: pure FDSE performs poorly, and gains are largest for coherent spatial fields where local order carries information about the underlying signal. These results suggest that, in compressed-latent reconstruction, appropriately weighted local-structure supervision can guide optimization toward solutions with better pointwise accuracy, rather than merely improving perceptual or structural metrics at MSE’s expense. 1 Introduction In autoencoders that reconstruct numerical arrays (tensors), reconstruction quality is captured by many non-pointwise criteria beyond mean squared error (MSE), including structural similarity [ 1 ] , finite-difference and gradient fidelity, spectral content, and topology-sensitive scalar-field diagnostics relevant to downstream scientific analysis [ 2 ] . These criteria are usually treated as objectives that trade off against MSE, motivating gradient-balancing, dynamic-weighting, and Pareto methods for conflicting losses [ 3 , 4 , 5 , 6 ] . We ask whether that trade-off is necessary for standard autoencoders that compress inputs into a smaller latent representation and reconstruct them to their original size. We evaluate autoencoders trained with MSE combined with standard structural-similarity, smoothness, gradient-domain, and frequency-domain terms, and introduce finite-difference sign error ( FDSE ), a local-order auxiliary. We show that, in the reconstruction autoencoders studied here, validation-selected auxiliary-trained models can improve structural metrics and validation MSE simultaneously. To test whether this is merely an implicit MSE step-size effect, we decompose each normalized auxiliary gradient into its MSE-parallel and MSE-orthogonal components at initialization and run projection-matched MSE-only controls. This diagnostic separates cases where an auxiliary mostly rescales the MSE update from cases where it contributes a directionally distinct initial update component. FDSE penalizes mismatches in the signs of adjacent finite differences between target and reconstruction, preserving whether each local difference is positive, negative, or zero. It requires no architecture changes and scales to arbitrary tensor ranks. Because exact sign matching is invariant to positive affine transformations, FDSE cannot control offset or amplitude by itself; throughout the paper it is paired with MSE. The practical question is whether this local-order supervision enables an MSE-trained reconstructor to reach lower validation MSE on structured scalar fields than pure MSE or alternative auxiliaries. Applicability. FDSE is intended for tasks where forward differences are taken along axes that index comparable states (consecutive grid points, time slices, or pressure levels), where training rewards fidelity to a fixed target under a pointwise term, and where the target varies coherently at the resolved scale rather than being texture-dominated. Empirical claim. Across four reconstruction tasks (shallow-water states, ERA5 potential-vorticity blocks, PDEBench radial-dam-break scalar fields, and CIFAR-10 images) using four autoencoder-task pairs (CRA5 [ 7 ] , NDAE , an MBT-style ND codec, and a factorized-prior image autoencoder), a moderate MSE: FDSE mixture reaches lower validation MSE than pure MSE in every pair, with reductions ranging from 2.3\times to 7.0\times . An ablation against several smoothing-, gradient-, and spectral-magnitude auxiliaries shows that, when auxiliaries improve structural features, this improvement is often coupled with improved MSE, and that FDSE is frequently among the strongest auxiliaries. All empirical claims are about validation-selected reconstruction quality; held-out deployment generalization is outside the scope of this investigation. Contributions. • We demonstrate a coupled relationship in compressed-latent reconstruction autoencoders: structural auxiliary losses can improve validation MSE while also improving non-pointwise structural metrics. • FDSE : a differentiable local-order auxiliary for MSE-trained reconstruction that penalizes sign mismatches in adjacent finite differences. • Empirical evidence on four reconstruction tasks and four autoencoder/task pairs that moderate MSE: FDSE mixtures lower validation MSE relative to pure MSE and are often competitive with or better than standard alternative auxiliaries. 2 Related Work FDSE relates to several existing approaches that move beyond pointwise fidelity. Classical regularizers such as total variation [ 8 ] , Tikhonov-style smoothness penalties [ 9 ] , and spectral fidelity terms [ 10 ] are typically added to a pointwise data-fidelity term, but they explicitly suppress high frequencies or enforce gradient sparsity rather than supervising local ordering. Gradient-domain and edge-aware losses [ 11 , 12 , 13 , 14 ] match exact finite-difference magnitudes or edge maps, whereas FDSE matches their signs . Image-restoration and modern tokenizer work show that autoencoder losses are often composite: perceptual, adversarial, semantic, and decoder-feature objectives are used to improve visual reconstructions or generative latents [ 1 , 15 , 16 , 17 , 18 , 19 ] . These objectives target perceptual realism, semantic compactness, or iterative decoding, while FDSE targets local finite-difference order in gridded tensors. Autoencoders learn representations that preserve the information rewarded by their training objective [ 20 , 21 , 22 ] . Learned compression makes this dependence explicit: deterministic and variational autoencoders are commonly trained under distortion, rate-distortion, or rate-distortion-perception objectives, and recent work studies how to balance these competing terms [ 23 , 24 , 25 ] . Our setting keeps the architecture fixed and asks a complementary question about the reconstruction objective itself: when an autoencoder is trained primarily for value fidelity, can adding a local-order auxiliary improve the trained reconstruction rather than trading MSE for another metric? In this sense, FDSE is an objective-level contribution that can be paired with different deterministic or variational autoencoder architectures [ 7 ] . Other autoencoder settings expose related objective tradeoffs. Sparse-autoencoder work for interpretability balances reconstruction against sparsity and dead-latent penalties [ 26 ] . Topology-aware regularizers [ 27 , 28 , 29 , 30 , 31 , 32 ] optimize latent geometry, batch-level point clouds, or mask topology rather than the local structure of each reconstructed tensor. Because FDSE is mixed with MSE, our work also intersects multi-task loss balancing [ 3 , 4 , 5 , 6 , 33 , 34 , 35 ] . That literature treats loss weighting and gradient conflict as substantive optimization problems, motivating uncertainty weighting, gradient normalization, Pareto methods, gradient surgery, and related dynamic or randomized weighting rules. Here, we treat the mixture weight as a validation-selected hyperparameter rather than proposing a new balancing rule. 3 FDSE : Local-Order Supervision for Reconstruction Pointwise reconstruction losses supervise individual entries. In a compressed-latent reconstructor for a coherent scalar field, locally wrong order usually means the model has displaced or flattened a ridge, trough, front, or monotone stretch, which can also raise pointwise error. For example, a local maximum is the location at which an increasing forward difference is