Efficient automatic design of robots

Significance

AI systems have taken massive strides designing prose, artwork, gameplay, software, and proteins but have yet to master the design of complex physical machines. Here we introduce an automatic optimization method that can design self-moving machines—robots—from scratch by tracing failures in their behavior back to errors or inefficiencies in particular parts of their physical structure. Because this method improves the robot in this way, it can optimize the interdependent parts of the robot much more rapidly than the current approach, in which the designer tries different robot designs in a trial-and-error fashion. This opens the way toward bespoke AI-driven design of robots for a wide range of tasks, rapidly and on demand.

Abstract

Robots are notoriously difficult to design because of complex interdependencies between their physical structure, sensory and motor layouts, and behavior. Despite this, almost every detail of every robot built to date has been manually determined by a human designer after several months or years of iterative ideation, prototyping, and testing. Inspired by evolutionary design in nature, the automated design of robots using evolutionary algorithms has been attempted for two decades, but it too remains inefficient: days of supercomputing are required to design robots in simulation that, when manufactured, exhibit desired behavior. Here we show de novo optimization of a robot’s structure to exhibit a desired behavior, within seconds on a single consumer-grade computer, and the manufactured robot’s retention of that behavior. Unlike other gradient-based robot design methods, this algorithm does not presuppose any particular anatomical form; starting instead from a randomly-generated apodous body plan, it consistently discovers legged locomotion, the most efficient known form of terrestrial movement. If combined with automated fabrication and scaled up to more challenging tasks, this advance promises near-instantaneous design, manufacture, and deployment of unique and useful machines for medical, environmental, vehicular, and space-based tasks.

Autonomous robots pale in comparison to the complexity, adaptivity, and diversity of their living counterparts, animals, which emerged and radiated over billions of years of evolution. While human minds can invent new mobile machines much faster than natural selection, the forms and functions of these artifacts are tightly constrained by human imagination as well as economics: considerable labor, time, money, and resources are required to design, test, and deploy a new robot for a new task. As a result, existing mobile robots tend to share canonical body shapes, such as quadrupedal or humanoid, canonical size scales, such as centimeters or meters in length, and canonical materials, such as metals and plastics. Despite this standardization, no proof exists that such robot body plans are the best fit for the robot’s intended niche. If scalable design and deployment of robots were to be achieved, and those robots could be rendered recyclable (1) or biodegradable (2), robots of diverse size and shape could be deployed into millions of niches in natural and built environments, much like evolution and reproduction does in nature.

Thus, here we demonstrate an alternative, scalable approach for rapidly and cost-effectively designing new kinds of robot body plans with minimal prior assumptions: a custom robot was efficiently and automatically generated in less than 30 s on a laptop computer (Fig. 1), requiring only 10 design attempts. This is orders of magnitude more efficient than all reported robot design methods that utilized computer simulations (1–10) and an order of magnitude fewer design attempts than a method that forewent simulation (11). Although many different approaches to the automatic design of virtual robots have been reported in the literature (12–30), the automatic design of physical robots to date has been achieved predominately through evolutionary algorithms: design variants evolve through random variation and selection. The trial-and-error nature of this process can lead to thousands or even millions of robots being evaluated in virtual environments before selecting one suitable for physical fabrication (1–10). Other evolutionary methods have evolved physical robots directly, without recourse to simulation, but these require hundreds of physical trials, again reducing efficiency (11). Evolutionary approaches are appealing for robot design as, like biological evolution, they often discover diverse and nonobvious body plans and behaviors; they cannot, however, guarantee discovery of locally optimal designs, and they are inefficient as the majority of effort is spent evaluating evolutionary dead ends. For these reasons, almost every other aspect of automatic optimization has found success replacing such trial-and-error search with gradient descent methods, which can efficiently follow paths through design space to optima.

Fig. 1.

Efficient automatic design. (A) The gradient-based approach introduced here takes 10 design attempts compared to thousands of simulations [blue dots in A; (1–10)] or hundreds of physical prototypes [orange dot in A; (11)] required by prior work. A mobile robot (B) was efficiently and automatically designed: starting from a randomly generated morphology (gray body shape with red musculature; C), nine design revisions (D–L) improve locomotive ability (green CoM traces). (M) The first, initially randomly configured robot (C) moves up and down in place, while its nine descendents (attempts 2 to 10; D–L) exhibit increasingly better locomotive ability (movement into the right-hand side of the page).

Applying gradient-based optimization to robot design has proven challenging because, like all motile organisms, there are complex feedback loops between the robot’s body plan (e.g., its shape and distribution of motors) and its behavior. Determining which aspect of a robot’s body plan caused some inefficiency in the robot’s behavior, and how that body part could be changed to improve behavior, is a version of the credit assignment problem, a ubiquitous problem in AI and, until now, unsolved for robot design.

The recent appearance of differentiable physical simulators (31, 32) has enabled gradient-based design of virtual robots (22, 27, 28): aspects of the robot’s shape or material properties responsible for poor behavior can be identified and mitigated nonrandomly to improve behavior. But the methods that produced them could not radically alter the robot’s internal structure (musculature, mass distribution, and voids) or external structure, such as the addition of new limbs. Moreover, none of them were realized as physical robots. Other gradient-based approaches have been reported for optimizing the orientation or length of a robot’s limbs (33–37), but the robot’s overall structure (its distribution of limbs and motors) was presupposed and held fixed. Thus, here we introduce an algorithm that (i) simulates and then assesses the fitness of a virtual robot’s behavior; (ii) identifies deficiencies in its overall shape, topology (number of voids), number and shape of limbs, mass distribution, musculature, and behavioral control; and (iii) changes all of them, simultaneously, so as to improve behavior in the next simulation. We demonstrate that only nine repetitions of this process (and thus only 10 simulations) provide a design that, when built, retains the optimized function (Fig. 2). This is orders of magnitude more efficient than state-of-the-art algorithms for automated design of physical robots (1–10).

Fig. 2.

Gradient-based optimization. (A) The robot was modeled as a two-dimensional plane of particles (gray), internal voids (circle; only one shown), and muscle patches (not shown). Voids reduce both the mass and elasticity of all particles within them. Particles closer to the center are reduced in mass/elasticity more; those very near the center are deleted entirely. (B) During simulation, each particle’s influence on the robot’s behavior is tracked. After simulation, this information is used to calculate how each particle should be changed to improve the robot’s behavior during the next simulation. In a simple example, such change is intuitive: If a robot incapable of moving should “move” to the right, the mass of particles to the left should be decreased (−) and those to the right should be increased (+). (C) These desired mass/elasticity changes dictate void movement: particles that should have less and more mass/elasticity pull and push on nearby voids, respectively. Voids respond to these pressures by moving (arrow) and resizing (adjusting their radius; not shown) and then dictate the new masses/elasticities of all particles within them; previously deleted particles may be recovered (dark gray). Using voids to indirectly affect particle masses/elasticities allows the design method to smoothly alter mass, elasticity, shape, and topology across the robot’s body. (D) Muscle patches (red) cause particles within them to volumetrically oscillate during simulation at the same phase, pushing and pulling against neighboring particles. As with voids, how the particles should have oscillated differently to improve behavior is calculated and used to move the muscle patches. (E) Sixty-four movable and resizable voids and 64 movable muscles were randomly positioned within the particle grid (Attempt 1), the resulting design was evaluated in simulation for locomotive ability, and the effect of each void and muscle on fitness was tracked. After simulation, how each void and muscle should be altered to improve fitness was calculated, and those alterations are made. The gradient of fitness with respect to each patch is followed for nine iterations (cyan to purple traces; F), resulting in an optimized design (Attempt 10; G). Once built, the randomly generated design (H) barely moves (I), whereas the optimized robot (J) exhibits legged locomotion.

Results

Terrestrial locomotion was selected as the target behavior as it is the defining function of higher animals. The fitness of a simulated robot design was thus defined as its mean velocity in the desired direction of travel. Starting from a randomly-generated body plan, fitness was increased from negligible movement to 0.5 body lengths per second (BL/s) (half of average human walking speed) with just nine additional design iterations (Fig. 1). We then conducted another 99 trials of this process, starting from 99 different randomly configured robots (SI Appendix, Fig. S2) and observed statistically significant increases in velocity from 0 to a mean of 0.45 BL/s over the 10 simulations, across these 100 trials (P < 0.01; Fig. 3). The final robot design from the first trial was built (SI Appendix, Fig. S1) and found to retain its behavior: when this physical optimized robot and another physical nonoptimized robot were each evaluated several times, the former moved on average significantly faster than the latter (P < 0.01; Fig. 2I).

Fig. 3.

Body shape and musculature. To better understand the gradients leading to better designs and their sensitivity to where in design space optimization starts, 99 additional independent runs of 10 design attempts each were performed, each starting from different random initial configurations. (A) Following gradients with respect to both body shape and musculature, simultaneously (blue) resulted in significantly higher fitness (P < 0.01) than optimizing body shape around a fixed musculature (orange; B) and optimizing musculature around a fixed body shape (green; C). Previous approaches to automate robot design were initialized with random samples of 10 or more designs; random samples of 10 designs yield little to no forward movement (purple; D). (A, Inset) The diversity of initial, random body plans before shape and musculature were co-optimized (light blue distribution) decreased significantly (P < 0.01; Mann–Whitney U test) as the body plans converged on similar, legged designs when optimization completed (dark blue distribution). (E) Internal voids tended either to move posteriorly (blue dot left of origin) or else shrink to zero radius and disappear, whereas muscles tended to move anteriorly (blue dot right of origin) as indicated by the heatmap of relative movement of muscles/voids from their initial locations (the origin). The result was tilted stacks of voids and muscles that form posterior-angled appendages with anterior musculature (F).

Several control experiments were performed to prove that the design process could and did make use of its freedom to change all aspects of the robot’s shape, topology, and musculature to improve behavior, despite the complex interdependencies between them (Fig. 3A). In the first control experiment, an initial random placement of muscles was generated and held fixed. Only the robot’s shape and topology were then optimized (Fig. 3B). In the second control experiment, muscle placement was optimized within a fixed, randomly generated shape and topology (Fig. 3C). The first and second control experiments yielded simulated robots with significantly lower fitness than the final robots from the original experiment [0.31 (99%CI 0.30 to 0.33) and 0.08 (99%CI 0.07 to 0.08) BL/s, respectively, compared to 0.45 (99%CI 0.43 to 0.47) BL/s], proving that body plan and musculature were consistently co-optimized. This demonstrates that automatic optimization methods must be allowed to simultaneously track and correct inefficiencies across many different aspects of a robot’s physical structure to achieve efficient design of motile behaviors. The demonstrated interdependency between robot shape, musculature, and movement also shows that robot design optimization is not reducible to the more commonly explored problem of topology optimization for static structures (38).

The algorithm reported here dominates state-of-the-art automated design of physical robots, which is currently accomplished with gradient-free, evolutionary algorithms. Since all such methods (Fig. 1A) start with populations of more than 10 randomly generated robots, they all reduce to random search when assigned a design budget of 10. We found that random search results in little to no appreciable forward movement (Fig. 3D) under these conditions.

Although our algorithm could have discovered crawling or peristaltic motion, it consistently rediscovered the third [and most efficient (39)] known form of terrestrial movement: legged locomotion (SI Appendix, Fig. S3). It did so by moving or merging internal voids to create gaps between prospective limbs (Fig. 3E and SI Appendix, Figs. S10–S13) and patterning muscles along each of their anterior edges to coordinate their movement (Fig. 3E). These changes consistently resulted in two or three posterior-angled appendages (Fig. 3F and SI Appendix, Fig. S3). When the algorithm was forced to work around a randomly generated and fixed musculature, it failed to discover legged locomotion (Fig. 3B).

Fig. 4 reports the approach’s generality: Robots can be automatically and efficiently designed under other constraints, such as conservation of materials (Fig. 4 A–D); for different tasks, such as object manipulation (Fig. 4E), object transport (Fig. 4F), and object ejection (Fig. 4G); and starting from different, arbitrary, nonrectangular initial shapes (Fig. 4 H–L). Within these new tasks and experimental conditions, very different kinds of robot forms and functions emerged.

Fig. 4.

Generality of results. The algorithm introduced here can be extended to design problems that require efficient use of material (e.g., maximizing speed while minimizing surface area; A–D); tasks other than locomotion, such as object manipulation (E), transport (F), and ejection (G); and may also begin from arbitrary, nonrectangular, initial boundary conditions such as triangles (H and I), a circle (J), a heptagram (K), or enneagram (nine-sided star; L).

Discussion

While these results demonstrate efficient design automation for a mobile robot, much of the design pipeline remains to be optimized. For instance, the fabrication process includes several manual steps that could be automated with embedded 3D printing (40), the computational efficiency of the physics model could be optimized to run faster than real time (41), and gradient computations could be optimized to be faster as well. Also, how the algorithm was allowed to alter the robot (moving and resizing internal voids or muscles) was somewhat arbitrary and may not readily extend to other kinds of materials (e.g., metals) and motors (e.g., rotary). Algorithms that are better at discovering and following gradients, and that generate robots from compressed representations (42) rather than directly rearranging their components, could likely be formulated. However, it is important to emphasize that gradient descent is inherently a local method and is therefore unlikely to find the globally optimal design—in the worst case, when designs are only of value if perfect, gradient descent will fail—but, in practice, gradient methods perform well in nonconvex search spaces, find approximate global optima (43), and can be redeployed (SI Appendix, Figs. S10–S13) from different random initial configurations (SI Appendix, Fig. S2). Finally, aspects of the physical robot’s design were improved against just one feature (speed) of one behavior (locomotion); in nature, diverse selection pressures simultaneously act on species. Thus, generalizing our approach such that alterations improve multiple features (reliability, safety, and energy efficiency) of multiple behaviors (object manipulation and collective behavior) remains unsolved but could in principle be achieved through multiobjective optimization (44) and optimizing the objective functions themselves (45). Despite these current limitations, the significant efficiency gains reported here compared to prior work suggests that this approach can be more easily scaled to realize physical robots with more complex morphologies and behaviors, such as those already observed in simulation (Fig. 4).

Three decades ago, gradient-based optimization methods were introduced which allowed for efficient automatic design of neural networks (46). These networks are now affecting human health (47), the economy (48), and the environment (49), albeit indirectly via human or machine intermediaries. Scaling robotics by automating robot design could have as much, if not more impact, because robots could work directly within unique environments ranging from cells to asteroids. However, this will only be possible if they have appropriate and tailored shapes, sizes, and material compositions to perform successfully and safely within them.

Materials and Methods

The design pipeline starts with a randomly configured simulated robot and optimizes its structure over nine design iterations. The final robot design is then built and its physical behavior is assessed.

Initial Conditions.

Sixty-four voids and 64 muscle patches are placed randomly within a 20-by-14-cm rectangular body. The initial location of each void was drawn from a uniform random distribution; the radius of each void was sampled from a Gaussian distribution with a mean of 0.91 cm and SD equal to the square of the mean. This corresponds to an initial void coverage (summed void areas; SI Appendix, section S2) of 60% of the robot’s body area, on average. The initial location of each muscle patch was drawn from a uniform random distribution. The radius of each muscle patch was fixed at 1.26 cm. This radius allows 64 nonoverlapping muscle patches to just cover the entire robot’s body with musculature, in case such a distribution, or one similar to it, is useful.

The robot was modeled as a collection of elastic particles (Fig. 2) using the Material Point Method (50), a hybrid Eulerian–Lagrangian model (SI Appendix, section S1). Attempts to directly optimize the parameters of individual particles, without explicit voids, did not yield any changes in the robot’s shape and resulted in significantly lower fitness (SI Appendix, Fig. S5).

Design.

Each design iteration comprises a forward and backward pass. During the forward pass, the robot is simulated using a differentiable simulator. At each time step of the simulation, internal forces from within the robot’s body (its muscles and elastic deformation) and external forces from the environment (gravity, friction, and collisions) are computed, and the acceleration of each particle is updated. At the end of the evaluation period, a fitness score was automatically assigned to grade the robot’s locomotive ability: the mean forward velocity of each particle during behavior, summed across particles. Since each function in this process is differentiable, the entire process, including the fitness computation, is differentiable.

During the backward pass, fitness affects the location and radius of each void (SI Appendix, Eq. S1 in SI Appendix, section S1) and the location of each muscle (SI Appendix, Eqs. S2 and S3 in SI Appendix, section S1). These functions are differentiable because they smoothly (quadratically) interpolate particle properties along the radius of an internal void or muscle, avoiding large discrete step functions at boundaries. More specifically, particle mass and elasticity decrease within a void, from edge to center, before entirely removing particles at their core (Fig. 2C and SI Appendix, Fig. S7C). Muscles similarly interpolate volumetric actuation, increasing actuation amplitude from edge to center. When a void is moved or decreased in radius, particles are gradually added back into the robot’s body (SI Appendix, section S1). Both musculature and voids were allowed to move off, and back on to, the body.

Physical Verification.

The physical robot is built by casting a silicone body with pneumatically actuated muscles (SI Appendix, section S3). To do so, two three-part molds of the design were 3D printed (SI Appendix, Fig. S1). Each mold was then filled with silicone and allowed to cure to form one-half of the robot’s body. The two halves (left and right) of the design were sealed together with additional silicone, and four retroreflective spheres were glued onto the top four corners to facilitate motion capture. Musculature was instantiated as a hollow bladder within the robot’s body and was actuated pneumatically through a dorsal air inlet.

The robot’s body and its environment were coated in cornstarch to match the friction parameters resident in the simulator. To compensate for asymmetries along the sagittal seam between the robot’s two sealed parts, the robot was placed between two alignment rails (Fig. 1B). This helps ensure that the robot’s movement stays along a straight, rather than curved, path. However, the rails are not strictly necessary (Movie S1).

The muscles were then actuated to create movement and allow for the possibility of locomotion in the desired direction. To do so, the robot’s air inlet was connected to a proportional pressure control valve that generated a square wave with an amplitude of 300 mBar, a wavelength of 500 ms, and a duty cycle of 50%. The robot was actuated for 1 min and its behavior was recorded with passive optical motion capture. After each behavior evaluation, the environment was reset: the robot was returned to its starting position, and additional cornstarch was added to the terrain and raked to have a thin grooved surface. Six behavioral trajectories of the optimized robot were collected, and five were collected of the unoptimized robot (Fig. 2I). The optimized design moved significantly further on average than the unoptimized design (P < 0.01; Mann–Whitney U test).

Generality.

Additional experiments were conducted in silico to better understand how (if at all) different random initial configurations (SI Appendix, Fig. S2) affect the optimized body plan (SI Appendix, Fig. S3), whether or not details of the manufacturing process could be incorporated as design constraints (SI Appendix, Fig. S4), how different encodings alter the design space (SI Appendix, Figs. S5 and S6), the algorithm’s sensitivity to hyperparameters (SI Appendix, Figs. S7–S9 and Table S2), and the paths gradient descent followed through morphospace (SI Appendix, Figs. S10–S13).

Supplementary Material

Appendix 01 (PDF)

Movie S1.

This movie summarizes the paper’s results (a novel robot is designed from scratch in just 10 design attempts) and methods (using gradient based optimization).

Data, Materials, and Software Availability

Source code is available in the GitHub repository (https://github.com/robodiff/robodiff) (51). All other data are included in the manuscript and/or supporting information.