Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control
Fabian Kresse, Christoph H. Lampert · 2025
Read Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control by Fabian Kresse, Christoph H. Lampert (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01467.txt.
Introduction
Fabian Kresse and Christoph Lampert ask if logic circuits can steer robots. Most control policies are neural nets with smooth weights. Circuits use yes-no gates and lookup tables. They are fast, tiny, and easy to check, but hard to train.
Their design, called DWCs, learns circuits in a smooth way. Real inputs are first coded as thermometer bits. Sparse gate layers then pick actions. The result is a policy made of pure logic. Logic policies could run on cheap chips for drones and tools where nets are too heavy. Circuits also shrug off weight noise, since a gate is right or wrong with no gray zone for small hardware faults to exploit.
Catalog record
| CREATOR | Fabian Kresse, Christoph H. Lampert |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01467
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control Fabian Kresse, Christoph H. Lampert Abstract We investigate whether continuous-control policies can be represented and learned as discrete logic circuits instead of continuous neural networks. We introduce Differentiable Weightless Controllers (DWCs), a symbolic-differentiable architecture that maps real-valued observations to actions using thermometer-encoded inputs, sparsely connected boolean lookup-table layers, and lightweight action heads. DWCs can be trained end-to-end by gradient-based techniques, yet compile directly into FPGA-compatible circuits with few- or even single-clock-cycle latency and nanojoule-level energy cost per action. Across five MuJoCo benchmarks, including high-dimensional Humanoid, DWCs achieve returns competitive with weight-based policies (full precision or quantized neural networks), matching performance on four tasks and isolating network capacity as the key limiting factor on HalfCheetah. Furthermore, DWCs exhibit structurally sparse and interpretable connectivity patterns, enabling a direct inspection of which input thresholds influence control decisions. 1 Introduction Deep Learning has transformed countless fields, from natural language processing ( 48 ; 40 ) and computer vision ( 29 ; 12 ) to game playing ( 36 ; 43 ) . The paradigm has successfully been transferred to continuous control as deep reinforcement learning (RL), where neural network policies serve as real-valued function approximators, trained to solve highly complex tasks, such as quadrotor racing ( 23 ) , robot locomotion ( 34 ) , and even fusion-plasma control ( 11 ) . Figure 1: Differentiable Weightless Controllers (DWCs): real-valued observations are thermometer-encoded into bitvectors, processed by two layers of multi-input boolean-output lookup tables (here drawn with 2 inputs), aggregated by group summation, and mapped via per-action memory lookups to final action values. However, policies implemented as neural networks rely on large numbers of compute-intensive multiply and accumulate operations, making their applications on resource-constrained hardware platforms, such as UAVs and mobile robots, difficult. In such cases, policies implemented as small and discrete-valued functions are preferable, as these allow efficient implementation on embedded hardware, such as FPGAs. Because low-precision floating point or integer-only operations have efficiency advantages even on high-end GPUs, several prior works have studied either converting standard-trained deep networks into a quantized representation (post-training compressions, PTQ), or learning deep networks directly in quantized form (quantization-aware training, QAT), see e.g. 16 for a recent survey. However, most such works target a static supervised or next-word-prediction setting, in which the input and output domains are discrete to start with, and the task involves predicting individual outputs for individual inputs. The setup of continuous control , i.e. learning control policies for cyber-physical systems such as autonomous robots or wearable devices, is more challenging, because input states and output actions can be continuous-valued instead of categorical, and the controller is meant to be run repeatedly over long durations, such that even small errors in the control signal might accumulate over time. Consequently, only few methods for quantized policy learning have been proposed so far. 28 discuss PTQ from a perspective of reducing resources during training in RL and, in QAT ablations, find that low-bit quantization often does not harm returns. 20 investigate both QAT and pruning (i.e. , reducing the number of weights) jointly, finding that the combination of high sparsity and 8-bit weight quantization does not reduce control performance. Very recently, 25 showed that controllers can be trained using QAT with neurons that have weights and internal activation values of only 2 or 3 bits; we evaluate our approach against this baseline in the experiment section. In this work, we challenge the necessity of relying on weight-based neural networks, i.e. , those that rely on large matrix multiplications during inference, for learning continuous control policies. Instead, we introduce DWCs (Differentiable Weightless Controllers) , a symbolic-differentiable architecture for continuous control in which dense matrix multiplication is replaced with sparse boolean logic. Figure 1 illustrates the inference pipeline. At their core lie differentiable weightless networks (DWNs) ( 6 ) , an FPGA-compatible and alternative variant of logic-gate networks (LGNs) ( 38 ) that were originally proposed for high-throughput, low-energy classification tasks. We extend the DWN architecture to the continuous RL domain, enabling synthesis of control policies that are digital circuits instead of real-valued functions. Specifically, we introduce a data-adaptive input encoding that converts real-valued input signals into binary vectors, and a trainable output decoding that converts binary output vectors into real-valued actions of suitable range and scale. The resulting DWCs are compatible with gradient-based RL, and we demonstrate experimentally that the learned DWCs match weight-based neural network baselines (full precision or quantized) on standard MuJoCo benchmarks, including the high-dimensional Humanoid task. At deployment time, DWCs process continuous observations by quantizing them using quantile binning and thermometer encoding, resulting in a fixed-width bitvector representation for each observation dimension. The bitvectors are concatenated and processed by a DWN, which consists of sparsely-connected layers of boolean-output lookup tables (LUTs) ( 6 ) . The final layer produces a fixed number of bit outputs per action dimension. These are summed using a popcount operation and converted to a final action value using a single-cycle (SRAM) memory lookup. As a consequence, DWCs are compatible with embedded hardware platforms, especially FPGA platforms that explicitly support LUT operations. Here, DWCs can run with few- or even single-cycle latency and minuscule (e.g. Nanojoule-level) energy per operation, as we demonstrate for the case of an AMD Xilinx Artix-7. Besides their efficiency, DWCs also have high potential for interpretability, because they cconsist of sparse, discrete, and symbolic elements instead of dense, continuous, matrix-multiplications in standard networks. For instance, the sparse connectivity in the first layer allows for direct identification of the input dimensions and thresholds utilized by the controller. Contributions. To summarize, our main contribution is the symbolic-differentiable DWC architecture that extends previous DWNs from classification to continuous control tasks. We demonstrate that • DWCs can be trained successfully using standard RL algorithms, reaching parity with floating-point policies (full precision or quantized) in most of our experiments. • DWCs are orders of magnitude more efficient than previously proposed architectures, achieving few- or even single-clock-cycle latency and nanojoule-level energy cost per action when compile to FPGA hardware. • DWCs allow for straightforward interpretation of some aspects of the policies they implement, specifically which input dimensions matter for the decisions, and what the relevant thresholds are. 2 Background We briefly review deep reinforcement learning for continuous control, and provide background details on DWNs. 2.1 Reinforcement Learning (RL) Reinforcement learning studies how an agent can learn, through trial and error, to maximize return (cumulative reward) while interacting with an environment ( 44 ) . There exist various reinforcement learning algorithms that are compatible with our setting of continuous control (continuous actions and observations). In the main body of this work we investigate the performance of DWCs with the state-of-the-art Soft-Actor Critic (SAC) method ( 18 ) . Results for Deep Deterministic Policy Gradient (DDPG) ( 31 ) and Proximal Policy Optimization (PPO) ( 41 ) can be found in Appendix 3 . SAC is an off-policy method, which keeps a buffer of previous state transitions and taken actions. It updates the parameters of the policy network based on this buffer with advantage estimations from two auxiliary networks (called critics ). Actions during training are sampled stochastically from a normal distribution, parametrized as