Young animals gallop across fields, climb trees and immediately find their feet with enviable grace after they fall1<\/a><\/sup>. And like our primate cousins, humans can deploy opposable thumbs and fine motor skills to complete tasks such as effortlessly peeling a clementine or feeling for the correct key in a dark hallway. Although walking and grasping are easy for many living things, robots have been notoriously poor at gaited locomotion and manual dexterity. Until now.<\/p>\n Writing in\u00a0Science Robotics<\/i>, Hwangbo\u00a0et al<\/i>.2<\/a><\/sup>\u00a0report intriguing evidence that a data-driven approach to designing robotic software could overcome a long-standing challenge in robotics and artificial-intelligence research called the simulation\u2013reality gap. For decades, roboticists have guided the limbs of robots using software that is built on a foundation of predictive, mathematical models, known as classical control theory. However, this method has proved ineffective when applied to the seemingly simple problem of guiding robotic limbs through the tasks of walking, climbing and grasping objects of various shapes.<\/p>\n A robot typically begins its life in simulation. When its guiding software performs well in the virtual world, that software is placed in a robotic body and then sent into the physical world. There, the robot will inevitably encounter limitless, and difficult to predict, irregularities in the environment. Examples of such issues include surface friction, structural flexibility, vibration, sensor delays and poorly timed actuators \u2014 devices that convert energy into movement. Unfortunately, these combined nuisances are impossible to describe fully, in advance, using mathematics. As a result, even a robot that performs beautifully in simulation will stumble and fall after a few encounters with seemingly minor physical obstacles.<\/p>\n Hwangbo\u00a0et al.<\/i>\u00a0have demonstrated a way of closing this performance gap by blending classical control theory with machine-learning techniques. The team began by designing a conventional mathematical model of a medium-dog-sized quadrupedal robot called ANYmal (Fig. 1). Next, they collected data from the actuators that guide the movements of the robot\u2019s limbs. They fed this information into several machine-learning systems known as neural networks to build a second model \u2014 one that could automatically predict the idiosyncratic movements of the AMYmal robot\u2019s limbs. Finally, the team inserted the trained neural networks into its first model and ran the hybrid model in simulation on a standard desktop computer.<\/p>\n <\/p>\n Figure 1 | The ANYmal robot.<\/b>\u00a0Hwangbo\u00a0et al.<\/i>2<\/a><\/sup>\u00a0report that a data-driven approach to designing robotic software can improve the locomotion skills of robots. They demonstrate their method using the ANYmal robot \u2014 a medium-dog-sized quadrupedal system.<\/span>Credit: ETH Zurich\/Daniel Winkler<\/p>\n<\/figcaption><\/figure>\n The hybrid simulator was faster and more accurate than a simulator that was based on analytical models. But more importantly, when a locomotion strategy was optimized in the hybrid simulator, and then transferred into the robot\u2019s body and tested in the physical world, it was as successful as it was in simulation. This long-overdue breakthrough signals the demise of the seemingly insurmountable simulation\u2013reality gap.<\/p>\n <\/p>\n <\/p>\n The approach used by Hwangbo\u00a0et al.<\/i>\u00a0hints at another major shift in the field of robotics. Hybrid models are the first step towards this change. The next step will be to retire analytical models altogether, in favour of machine-learning models that are trained using data collected from a robot\u2019s real-world environment. Such data-pure approaches \u2014 referred to as end-to-end training \u2014 are gaining momentum. Several innovative applications have already been reported, including articulated robotic arms3<\/a><\/sup>, multi-fingered mechanical hands4<\/a><\/sup>, drones5<\/a><\/sup>\u00a0and even self-driving cars6<\/a><\/sup>.<\/p>\n For now, roboticists are still learning to harness the power of faster computation, an abundance of sensor data and improvements in the quality of machine-learning algorithms. It is not yet clear whether it is time for universities to stop teaching classical control theory. However, I think that the writing is already on the wall: future roboticists will no longer tell robots how to walk. Instead, they will let robots learn on their own, using data that are collected from their own bodies.<\/p>\n Many challenges remain, of course, and chief among them is the challenge of scalability. So far, end-to-end training has been applied to physical robots that have only a small number of actuators. The fewer the actuators, the fewer the parameters that are needed to describe the robot\u2019s movements, and therefore the simpler the model is. The path to scalability will probably involve the use of more-hierarchical and modular machine-learning architectures. Further research needs to be done to learn whether end-to-end control can be scaled up to guide complex machines that have dozens of actuators, including humanoid robots7<\/a><\/sup>, or large systems such as manufacturing plants or smart cities \u2014 urban areas that use digital technology to improve the lives of citizens.<\/p>\n Another challenge is less technical and more personal. For some researchers, the transition from using relatively simple mathematical models to applying \u2018black box\u2019 machine-learning systems \u2014 in which the internal workings are unknown \u2014 signals the unfortunate end of insight, and brings with it the feeling of loss of control. I am not one of those researchers. For me, there is something satisfying about seeing a robot, like a child, learn to walk on its own.<\/p>\n The insights offered by Hwangbo\u00a0et al.<\/i>\u00a0could also be considered in the context of the mysteries of the mind. Consciousness has been one of the longest-standing puzzles of human nature8<\/a><\/sup>. In my experience, human-devised definitions of self-awareness are so vague that they are of little practical value for building robotic software. Perhaps the converse is true, however, and the study of robotic software can offer insights into age-old questions about the human mind.<\/p>\n One could conjecture that self-awareness and, by extension, consciousness are, at their core, an indication of our ability to think about ourselves in the abstract \u2014 to self-simulate. I would argue that the further ahead in time a person can look, and the more detailed the mental picture of their future activities is, the greater that person\u2019s capacity for self-awareness will be. Now, robots are capable of learning to self-simulate. This breakthrough is not merely a practical advance that will save some engineering effort, but also the beginning of an era of robot autonomy.<\/p>\n <\/p>\n <\/p>\n<\/div>\n <\/p>\n (\uc6d0\ubb38: \uc5ec\uae30<\/a>\ub97c \ud074\ub9ad\ud558\uc138\uc694~)<\/p>\n <\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":"![\"ANYmal](\"\/\/media.nature.com\/w800\/magazine-assets\/d41586-019-00999-w\/d41586-019-00999-w_16581342.jpg\")
<\/div>\n<\/div>