Fare : Failure Resilience in Learned Visual Navigation Control
Zishuo Wang, Joel Loo, David Hsu · 2025
Read Fare : Failure Resilience in Learned Visual Navigation Control by Zishuo Wang, Joel Loo, David Hsu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24680.txt.
Introduction
Zishuo Wang, Joel Loo, and David Hsu want sight based robot guides that can bounce back from faults. Guides trained by copy can fail in odd new places, with no clear cause. Their frame, InFeR, adds fault sense and fix steps with no fault demos at all.
First, retraining shapes the inner code of the guide so odd cases stand out. Next, a heat map of the view points to the spot that most drove the fault call. A simple fix rule then acts on that clue. In real world tests, guides with this frame found and fixed faults on long runs.
Catalog record
| CREATOR | Zishuo Wang, Joel Loo, David Hsu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-24680
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fare : Failure Resilience in Learned Visual Navigation Control Zishuo Wang, Joel Loo, David Hsu Abstract While imitation learning (IL) enables effective visual navigation, IL policies are prone to unpredictable failures in out-of-distribution (OOD) scenarios. We advance the notion of failure-resilient policies , which not only detect failures but also recover from them automatically. Failure recognition that identifies the factors causing failure is key to informing recovery: e.g. pinpointing image regions triggering failure detections can provide cues to guide recovery. We present Fare , a framework to construct failure-resilient IL policies, embedding OOD-detection and recognition in them without using explicit failure data, and pairing them with recovery heuristics. Real-world experiments show that Fare enables failure recovery across two different policy architectures, enabling robust long-range navigation in complex environments. I Introduction Visual navigation is an attractive approach to robot navigation, leveraging rich visual information from low-cost sensors [ 1 ] . Imitation learning (IL) has emerged as a key method to learn visual navigation policies [ 2 , 3 , 4 ] , but is inherently limited by training data. IL policies may fail unpredictably on inputs outside the training distribution, often without clear explanation [ 5 , 6 , 7 ] . This work develops a mechanism to enable IL policies to detect and recover from failures, supporting robust open-world navigation. We argue that failure resilience in learned visual navigation extends beyond detecting failures, to also encompass automatic recovery from failures. Current approaches fall short in two ways. Firstly, they focus primarily on detecting failures, to trigger and learn from operator interventions [ 8 ] . We contend that the ability to independently recover is needed for greater autonomy, and that recognising the causes of failure is key to guiding recovery. Figure 1 shows examples of informed recovery : e.g. , given dynamic obstacles unseen in training, a resilient policy not only detects a failure but also recognises and highlights the moving person as the source of failure. This informs recovery actions, enabling the robot to avoid collisions and return to navigating. Secondly, existing methods may fail to capture policies’ true failure modes, as they often rely on failure detectors external to the policy [ 9 ] . This limits their ability to accurately detect failures and recover the robot to states in which the policy performs well. We argue that failure detection and recognition should be embedded into policies, and also unified to ensure detected failures are aligned with recognised causes. Fig. 1: Fare enables automatic recovery from policy failures. Limited training data causes failures on OOD scenarios, e.g. sensor failures, close dynamic obstacles etc. Fare not only detects these failures but also recognises failure causes in the input image, enabling informed recovery. These motivate two desiderata for failure-resilient policies: (i) informed recovery guided by failure recognition, and (ii) unified failure detection and recognition embedded within policies. We incorporate these in Fare , a novel framework for building failure-resilient visual navigation policies. Following prior work, we frame failures as scenarios out-of-distribution (OOD) for the policy [ 10 , 6 , 11 , 7 ] . Fare augments and retrains IL policies into OOD-aware policies—embedding OOD detection and recognition into the policy’s latent space—then pairs them with informed recovery strategies. OOD recognition is defined as the task of identifying input image regions triggering OOD detections, providing spatial feedback to select corrective actions for recovery. Fare achieves OOD-awareness without explicit failure data, by shaping the latent space to separate ID and OOD inputs. We show in the real world that Fare enables two different policies—DECISION [ 2 ] and the General Navigation Model (GNM) [ 3 ] to detect and recover from critical failures, allowing a robot to complete a challenging 300m indoor/outdoor route without intervention. II Related Work II-A Learned visual navigation Navigating purely with high-dimensional image inputs is challenging for classical robotic systems [ 1 ] . Recent progress in visual navigation is enabled by learning perceptual modules [ 12 , 13 ] or even entire policies end-to-end [ 2 , 3 , 4 ] . Among these, end-to-end imitation learning (IL) has proven especially effective [ 2 , 3 , 4 ] , enabling robust navigation in the wild [ 14 , 15 ] . However, such policies remain opaque and offer no indication of impending failure [ 9 ] . We address this by defining failure resilience for end-to-end IL policies and proposing Fare , a framework to augment IL policies into failure-resilient policies. II-B Failure detection Robust visual navigation in open-world settings hinges on the ability to detect failures [ 16 ] . Some recent approaches learn from explicit failure data: BADGR [ 17 ] heuristically labels failures in collected training data post hoc, while others learn from human interventions that precede failures [ 18 , 19 , 20 ] . Notably, [ 20 ] combines this with novelty detection to capture both user-defined failures and OOD events. Since imitation learning (IL) datasets typically include only successful trajectories, unsupervised failure detection without explicit failure data becomes crucial when learning policies with IL. Failure detection is often framed as identifying OOD inputs that lie outside the IL dataset’s distribution [ 6 ] . This can be approached via epistemic uncertainty estimation [ 21 ] using ensembles or Bayesian neural networks [ 9 ] , but such methods are often too computationally intensive for real-time navigation with limited compute [ 22 ] . Alternative methods seek greater efficiency, often at the cost of using detectors external to the policy which may not fully capture the its failure modes and tend toward overly conservative detections [ 9 ] . Autoencoders [ 6 , 7 ] and density estimators [ 23 , 24 ] are common approaches, but may not accurately capture the data distribution as they are trained on observations and exclude action data, often producing overly conservative detections [ 9 ] . [ 25 ] apply Random Network Distillation to learn an external detector for the joint observation-action distribution. FAIL-Detect [ 22 ] compares a range of OOD scoring approaches, and combines them with conformal prediction for OOD detection with statistical guarantees. Finally, some methods leverage LLMs to identify and explain failures [ 26 , 27 ] , though these lack efficiency and insight into the policy’s specific failure modes. Fare focuses on efficiency and on capturing the policy’s true failure modes, achieved by embedding failure detection within the IL policy. II-C Failure recovery Robust navigation requires not only the ability to detect failures, but also to independently recover from them. Existing approaches largely terminate the episode on detection, and seek human intervention [ 28 , 9 , 8 ] . A small subset of works attempt to autonomously recover from failure by executing predefined actions open-loop— e.g. returning to a default state [ 7 ] or executing a default behaviour [ 6 ] . Some systems layer increasingly aggressive preset actions [ 29 ] to improve recovery chances. [ 30 ] further requests interventions and learns from subsequent recovery demonstrations. Fare emphasises robustness through autonomous recovery and highlights the importance of OOD recognition in providing cues for informed, efficient recovery. To build failure-resilient policies, Fare combines OOD-aware