FaceSleuth: Learning-Driven Single-Orientation Attention Verifies Vertical Dominance in Micro-Expression Recognition
Linquan Wu, Tianxiang Jiang, Wenhao Duan, Jacky Keung, Yini Fang · 2025
Read FaceSleuth: Learning-Driven Single-Orientation Attention Verifies Vertical Dominance in Micro-Expression Recognition by Linquan Wu, Tianxiang Jiang, Wenhao Duan, Jacky Keung, Yini Fang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02695.txt.
Introduction
Linquan Wu and co-authors read the faint face twitches that flash in a blink, in a study from 2025. Such micro-moves can leak true feelings, but they are hard to spot and harder to grade on fresh faces. Stock tools lean on looks that shift from set to set, so scores crash in the wild.
Their frame, FaceSleuth-R, hunts motion paths more than looks. A light part learns, layer by layer, the best slant to watch. Across sets, that slant came out near straight up and down, a shared mark of such twitches. In hard tests that hold out a whole set, the frame beat past tools by a clear gap. It also set top scores on stock sets. The team says slant-wise focus is the key to wild use.
Catalog record
| CREATOR | Linquan Wu, Tianxiang Jiang, Wenhao Duan, Jacky Keung, Yini Fang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02695
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FaceSleuth: Learning-Driven Single-Orientation Attention Verifies Vertical Dominance in Micro-Expression Recognition Linquan Wu, Tianxiang Jiang, Wenhao Duan, Jacky Keung, Yini Fang Abstract Micro-expression recognition (MER) demands models that can amplify millisecond-level, low-amplitude facial motions while suppressing identity-specific appearance. We introduce FaceSleuth , a dual-stream architecture that (1) enhances motion along the empirically dominant vertical axis through a Continuously Vertical Attention (CVA) block, (2) localises the resulting signals with a Facial Position Focalizer built on hierarchical cross-window attention, and (3) steers feature learning toward physiologically meaningful regions via lightweight Action-Unit embeddings.To examine whether the hand-chosen vertical axis is indeed optimal, we further propose a Single-Orientation Attention (SOA) module that learns its own pooling direction end-to-end. SOA is differentiable, adds only 0.16 % parameters, and collapses to CVA when the learned angle converges to π / 2 \pi/2 . In practice, SOA reliably drifts to ∼ 88 ∘ \sim 88^{\circ} , confirming the effectiveness of the vertical prior while delivering consistent gains.On three standard MER benchmarks, FaceSleuth with CVA already surpasses previous state-of-the-art methods; plugging in SOA lifts accuracy and F1 score performance to 95.1 % / 0.918 on CASME II, 87.1 % / 0.840 on SAMM, and 92.9 % / 0.917 on MMEW without sacrificing model compactness. These results establish a new state of the art and, for the first time, provide empirical evidence that the vertical attention bias is the most discriminative orientation for MER. 1 Introduction Micro-expressions (MEs) are involuntary, millisecond-long facial muscle activations that betray genuine emotions people attempt to hide [ 1 ] . Each ME passes through onset , apex , and offset phases before the face returns to neutral appearance, and their diagnostic value has been leveraged in psychotherapy [ 2 ] , criminal investigation, and national security [ 3 ] . The combination of extremely short duration, low motion amplitude, and strong person-specific appearance makes automatic micro-expression recognition (MER) one of the most challenging fine-grained video-analysis tasks. Limitations of existing deep models. Early CNN-based pipelines boosted MER accuracy by learning hierarchical appearance-motion features [ 7 , 8 , 9 ] . Yet CNN receptive fields grow slowly, and the models risk over-fitting identity cues that dominate static frames. Vision Transformer (ViT) variants such as MMNet [ 10 ] alleviate this bias by splitting motion and location streams, but single-frame positional embeddings discard temporal context and the dense global attention struggles to model the long-range dependencies required for ME localisation. Vertical motion prior & open question. Recent psychophysiological evidence shows that the most informative facial muscle trajectories for MER travel predominantly vertically —e.g. brow raises or lip presses—rather than horizontally. We therefore introduced a Continuously Vertical Attention (CVA) block that amplifies motion along the y -axis inside a dual-stream network called FaceSleuth . While CVA improves recognition, it also raises a methodological question: Is the vertical axis truly the globally optimal direction, or merely a helpful hand-crafted prior that might be sub-optimal for some identities or datasets? This work. To answer that question without sacrificing the benefits of CVA, we add a lightweight Single-Orientation Attention (SOA) module. SOA keeps the pooling-along-lines philosophy of CVA but learns the most discriminative axis \theta\!\in\![0,\pi) end-to-end; if the learned angle converges to \tfrac{\pi}{2} , SOA degenerates to the original CVA, otherwise it can adapt to oblique or horizontal motion patterns. Combined with a Facial Position Focalizer (FPF) based on the Swin Transformer and Action-Unit (AU) embeddings that further suppress identity noise, the resulting FaceSleuth framework captures subtle spatio-temporal cues with minimal overhead. Contributions. • We propose FaceSleuth , a dual-stream MER network that couples a Continuously Vertical Attention block with a Facial Position Focalizer and AU embeddings, yielding strong spatial and temporal discrimination. • We introduce Single-Orientation Attention (SOA), the first differentiable pooling operator that learns a single dominant axis. SOA generalises CVA and—through gradient-based optimisation—empirically verifies that the vertical direction is indeed the best inductive bias for MER. • Extensive experiments on CASME II, SAMM and MMEW show that FaceSleuth with SOA sets a new state of the art (e.g. 95.1 % accuracy / 0.918 F1 on CASME II) while adding only 0.16% parameters. Ablation studies (Secs. 3.4–3.5) quantify CVA’s benefit and demonstrate SOA’s convergence to an \approx 88^{\circ} axis, confirming the vertical prior. Figure 1: The diagram of our proposed model(a) and the comparison between our CVA module (b) with our SOA module (c). 2 Related Work 2.1 Appearance–Motion Representation in MER Early automatic MER systems relied on optical flow [ 13 ] , Local Binary Pattern variants, or 3D-CNNs to encode subtle motion directly from frame sequences. Although effective on small benchmarks, these appearance-centred pipelines often learn subject-specific texture and illumination cues that hamper generalisation. 2.2 Action-Unit (AU) Assisted Recognition To inject domain priors, several works integrate muscle Action Units. Xie et al. [ 11 ] jointly performed AU detection and MER by building relational graphs among AU regions; the auxiliary task improved F1 but the Graph Attention CNN struggled on scarce MER data. Subsequent studies incorporate handcrafted AU masks as spatial priors, yet most treat AU labels as static inputs and do not explicitly model their relation to temporal motion. 2.3 Frame Selection and Dual-Input Designs Liong et al. [ 13 ] showed that using only the onset–apex pair can approximate sequence-level performance while avoiding complex temporal models. This two-frame paradigm motivates recent dual-stream networks that feed onset, apex, or flow together with auxiliary cues (e.g. AU masks) to the backbone, reducing redundancy and training cost. 2.4 Spatial Attention and Identity Leakage Channel–spatial attention modules such as CBAM have been transferred from generic image recognition to MER (e.g. MMNet [ 10 ] ) to highlight moving regions. Because CBAM computes global pooling before attention, identity features (skin tone, eyewear) may still dominate the weighting. Moreover, psychophysiological studies report that the most discriminative micro-movements are predominantly vertical (eyebrow raises, lip presses), hinting that a direction-specific inductive bias could offer cleaner separation; existing attention blocks remain agnostic to this property. 2.5 Facial Geometry Modelling with Graph Convolution Key-point graphs processed by GCNs [ 24 ] encode fine geometry at negligible pixel cost. However, vanilla GCNs aggregate information only from immediate neighbours, making it difficult to capture cross-region interactions such as coupled brow–mouth motions that often characterise MEs. Multi-hop or attention-based graph variants partly mitigate this but add depth and parameters that small MER sets cannot sustain. 2.6 Transformer Backbones for MER Vision Transformer (ViT) [ 18 ] has been adopted as a generic feature extractor in MMNet, yet its fixed patch partition and absolute position embeddings ignore the well-known hierarchy of facial scales. Swin Transformer [ 19 ] addresses this issue with a hierarchical pyramid and shifted windows, enabling both local focus and cross-window communication while keeping computation tractable—properties desirable for MER where subtle motion spans multiple granularities. 2.7 Summary and Gap In summary, prior research has (i) introduced AU priors, (ii) adopted sparsified frame inputs, and (iii) explored attention or transformer backbones, but it has not answered whether a hand-crafted vertical bias is indeed optimal, nor provided a principled mechanism able to learn and verify that bias on the fly. Our work fills this gap by coupling a direction-adaptive pooling module with hierarchical spatial modelling, as detailed