Generating Freeform Endoskeletal Robots
Muhan Li, Lingji Kong, Sam Kriegman · 2024
Read Generating Freeform Endoskeletal Robots by Muhan Li, Lingji Kong, Sam Kriegman (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-01036.txt.
Introduction
Muhan Li, Lingji Kong, and Sam Kriegman generate, in a 2024 paper, robots with soft bodies wrapped around jointed inner skeletons. Robot design by computer is decades old, yet it has stuck to two extremes: all-rigid machines with joints, and all-soft blobs without them. Animals, of course, mix both.
Their system builds body plans from elastic and rigid cells joined directly together. Designs are encoded in a smooth latent space where search can roam. Each decoded robot is then tuned for how it senses and moves. Endoskeletons borrow evolution's trick: soft tissue for contact, bone for leverage. Evolving inside and outside at once opens body plans no engineer would sketch.
Catalog record
| CREATOR | Muhan Li, Lingji Kong, Sam Kriegman |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-01036
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generating Freeform Endoskeletal Robots Muhan Li, Lingji Kong, Sam Kriegman Abstract The automatic design of embodied agents (e.g. robots) has existed for 31 years and is experiencing a renaissance of interest in the literature. To date however, the field has remained narrowly focused on two kinds of anatomically simple robots: (1) fully rigid, jointed bodies; and (2) fully soft, jointless bodies. Here we bridge these two extremes with the open ended creation of terrestrial endoskeletal robots: deformable soft bodies that leverage jointed internal skeletons to move efficiently across land. Simultaneous de novo generation of external and internal structures is achieved by (i) modeling 3D endoskeletal body plans as integrated collections of elastic and rigid cells that directly attach to form soft tissues anchored to compound rigid bodies; (ii) encoding these discrete mechanical subsystems into a continuous yet coherent latent embedding; (iii) optimizing the sensorimotor coordination of each decoded design using model-free reinforcement learning; and (iv) navigating this smooth yet highly non-convex latent manifold using evolutionary strategies. This yields an endless stream of novel species of “higher robots” that, like all higher animals, harness the mechanical advantages of both elastic tissues and skeletal levers for terrestrial travel. It also provides a plug-and-play experimental platform for benchmarking evolutionary design and representation learning algorithms in complex hierarchical embodied systems. Figure 1: An evolving population of endoskeletal soft robots were encoded by a low dimensional latent design genome with minimal morphological assumptions and optimized for locomotion across complex terrains in multiphysics simulation using a shared universal controller that was simultaneously learned alongside morphological design. Videos and code at https://endoskeletal.github.io . Figure 2: Designing endoskeletal robots. A population of 64 robot designs ( A-C ) was sampled from a multivariate normal distribution ( D ) centered about an initially random point in latent space. The behavior each design in the population was optimized in physics simulation for 20 epochs of reinforcement learning under a shared universal controller. The ten red behavioral traces to the right of A-C project the forward locomotion of these randomly generated designs (from 3D to 2D) into the right hand side of the page, at every other epoch of learning. The fitness of a design was based on the best performance it achieved during reinforcement learning. After learning, the mean and covariance matrix of the design distribution were then adapted by evolutionary strategies and a subsequent generation of 64 new designs was sampled from the shifted distribution ( E ). This process was repeated for dozens of generations ( F-J ) yielding an evolved design distribution ( K ) that encodes a population of designs ( L-N ) with much higher fitness (blue behavioral traces). Figure 3: A universal controller for freeform endoskeletal robots. The control of an endoskeletal robot ( A ) utilizes a graph data structure with edges that map bone connections within its skeleton, edge features that track the position, orientation and angle of each joint, and node features that track the position, orientation and movement of each bone ( B ). The controller also tracks proprioceptive and mechanosensory input from the position, velocity and strain of the robot’s soft tissues ( C ), which is locally pooled into the center of mass of each bone ( D ) thereby transforming the soft tissue’s sensory input from voxel space to a graph that aligns with the skeletal sensory graph ( E ). The combined rigid+soft sensory graph is then fed as input to a graph transformer ( F ). The graph transformer distils sensory signatures across the graph into updated node features ( G ), and updated edge features ( H ). The Actor ( I ) takes the updated edge features as input and outputs motor commands ( J ): the target rotation angle for each joint ( K ). The updated node features are pooled by globally by a channel-wise maximum across the node dimension ( L ) to retrieve a graph-level output ( M ), which the Critic ( N ) uses to predict a value function based on the robot’s current state information. 1 Introduction The manual synthesis of a single robot for a single task requires years if not decades of labor-intensive R&D from teams of engineers who rely heavily on expert knowledge, intuition, and hard-earned experience. The automatic synthesis of robots may reveal designs that are different from or beyond what human engineers were previously capable of imagining and would be of great use if robots are to be designed and deployed for many disparate tasks in society. However, automating the creation and improvement of a robot’s design is notoriously difficult due to the hierarchical and combinatorial nature of the optimization problem: the objective and variables of the robot’s controller optimization depend on the optimizer of the robot’s discrete mechanical structure (e.g. its distribution of motors and limbs). Considerable effort has been and is being expended to ameliorate ( Zhao et al., 2020 ; Yuan et al., 2022 ; Strgar et al., 2024 ) or obviate ( Hejna et al., 2021 ; Gupta et al., 2022 ; Matthews et al., 2023 ) these technical challenges. Nevertheless, automatically designed robots have yet to evolve beyond primitive machines with exceedingly simple body plans, which can be neatly divided into two opposing groups: 1. fully rigid, jointed bodies ( Sims, 1994 ; Lipson & Pollack, 2000 ; Auerbach & Bongard, 2014 ; Zhao et al., 2020 ; Hejna et al., 2021 ; Gupta et al., 2021 ; Yuan et al., 2022 ) ; and 2. fully soft, jointless bodies ( Hiller & Lipson, 2012 ; Cheney et al., 2018 ; Kriegman et al., 2020a ; Matthews et al., 2023 ; Wang et al., 2023b ; Li et al., 2024 ; Strgar et al., 2024 ) . Here we introduce the de novo optimization of externally soft yet internally rigid, jointed body plans—endoskeletal robots (Fig. 1 )—with minimal assumptions about the robots’ external and internal structures (Fig. 2 ). Endoskeletal robots contain bones and joints—the lever systems animals use to their mechanical advantage as they gallop across uneven surfaces, climb up trees, and swing from branch to branch. At the same time, endoskeletal robots contain and are encased by soft deformable tissues—the springs animals use to conform to uneven surfaces, absorb shocks, store and release elastic energy, catapult themselves into the air, and capture complex somatosensory signals about their environment. In order to achieve this confluence of soft and rigid body dynamics within a single robot—without presupposing the topology or the geometry of the robot’s bones and tissues—a novel multi-physics voxel-based simulator (Fig. 4 ) was created and is introduced here alongside a latent endoskeletal embedding and companion generator that sythensizes elastic and rigid voxel cells into freeform tissues and bones, and connects the bones by joints within a functional whole. Several existing simulators support two-way coupling between rigid and deformable bodies with joint constraints ( Shinar et al., 2008 ; Kim & Pollard, 2011 ; Li et al., 2020 ) . However they are not naturally amenable to structural optimization because they do not readily expose building blocks (e.g. voxels) to the optimizer; instead, they require a bespoke volumetric mesh to be supplied for each