Watch this robot dog scramble over tricky terrain just by using its camera

Watch this robot dog scramble over tricky terrain just by using its camera thumbnail

When Ananye Agarwal took his dog for a walk in the local park near Carnegie Mellon University’s campus, other dogs stopped dead in their tracks.

This is because Agarwal’s dog, Agarwal, was a robot. His robot uses a built in camera, unlike other robots that rely heavily upon an internal map to navigate. Agarwal is a PhD student at Carnegie Mellon. He has developed a technique that allows robots to navigate on difficult terrain using reinforcement learning and computer vision. Their work is expected to make it easier for robots in the real world.

Unlike existing robots on the market, such as Boston Dynamics’ Spot, which moves around using internal maps, this robot uses cameras alone to guide its movements in the wild, says Ashish Kumar, a graduate student at UC Berkeley, who is one of the authors of a paper describing the work; it’s due to be presented at the Conference on Robot Learning next month. While other attempts to use cameras to guide robot movements have been limited to flat terrain and not much else, they were able to get their robot up stairs, onto stones, and over gaps.

grid of clips of robot dog walking on stairs


The four-legged robot is first trained to move around different environments in a simulator, so it has a general idea of what walking in a park or up and down stairs is like. The robot’s movement is guided by visuals from the single camera at its front. Reinforcement learning is an AI technique that allows systems improve through trial-and-error. This robot adjusts its gait to navigate stairs and uneven terrain.

Removing an internal map makes the robot less vulnerable to errors in the map, says Deepak Pathak (an assistant professor at Carnegie Mellon), who was part the team.

It is very difficult for a robot, to translate raw pixels from the camera into the precise and balanced movement required to navigate its surroundings. Jie Tan, a Google research scientist, was not involved in this study. He said that this is the first time that a small, low-cost robot has demonstrated such remarkable mobility.

The team has achieved a breakthrough in robot learning, autonomy, says Guanya Shi, an academic at the University of Washington who studies robotic control and machine learning.

Akshararai, a researcher at Facebook AI Research, who works on machine-learning and robotics, agrees.

This work is a promising step towards building such perceptive-legged robots and deploying them into the wild,” Rai says. Rai states that while the work of the team is useful in improving the robot’s walking, it will not help the robot decide where to go. She says that navigation is crucial for robot deployments in the real world.

It will take more work before the robot can move around the house or fetch things. Tan states that while the robot can see depth through its front camera, it is unable to handle situations like slippery ground or tall grass. It could also get stuck in mud or puddles.

Read More