Researchers empower robot to fold towels

Who wouldn’t want a robot that could make your bed or do the laundry? A team of Berkeley researchers has brought us one important step closer by, for the first time, enabling an autonomous robot to reliably fold piles of previously unseen towels.

Robots that can do things like assembling cars have been around for decades. The towel-folding robot, however, is doing something very new, according to the researchers, doctoral student Jeremy Maitin-Shepard and assistant professor Pieter Abbeel, both of UC Berkeley’s Department of Electrical Engineering and Computer Sciences.

Robots like those who assemble cars are designed to work in highly structured settings that allow them to perform a wide variety of tasks with mind-boggling precision and repeatability—but only in these carefully controlled environments, Maitin-Shepard and Abbeel explain. Outside of such settings, their capabilities are much more limited.

Automation of household tasks like folding laundry is somewhat compelling in itself. But more significantly, according to Maitin-Shepard, it involves a task that’s proved a challenge for robots: perceiving and manipulating “deformable objects”—things that are flexible, not rigid, so their shape isn’t predictable. A towel is deformable; a mug or a computer isn’t.

The video of the robot in action tells the story best. It features a robot built by robotics research laboratory Willow Garage, running an algorithm developed by the Berkeley team and faced with a heap of towels it has never “seen” before. The towels are of different sizes, colors and materials. The robot picks one up and turns it slowly, first with one arm and then with the other. It uses a pair of high-resolution cameras to scan the towel to estimate its shape. Once it finds two adjacent corners, it can start folding. On a flat surface, it completes the folds, smoothing the towel after each fold and making a neat stack.

“Existing work on robotic laundry and towel folding has shown that, starting from a known configuration, the actual folding can be performed using standard techniques in robotic manufacturing,” Maitin-Shepard says.

TEACHERS OF MACHINES: From left, assistant professor Pieter Abbeel and doctoral student Jeremy Maitin-Shepard of UC Berkeley’s Department of Electrical Engineering and Computer Sciences led the research on the laundry-folding robot. (Photo by the researchers.)

But there’s been a bottleneck: getting a towel picked up from a pile where its configuration is unknown and arbitrary, and turning it into a known, predictable shape. That’s because existing computer-vision techniques, which were primarily developed for rigid objects, aren’t robust enough to handle possible variations in three-dimensional shape, appearance and texture that can occur with deformable objects, the researchers say.

Solving that problem helps a robot fold towels. But more significantly, it addresses a key issue in the development of robotics.

“Many important problems in robotics and computer vision involve deformable objects,” Abbeel says. “The challenges posed by robotic towel-folding reflect important challenges inherent in robotic perception and manipulation for deformable objects.”

The team’s technical innovation is a new computer vision–based approach for detecting the key points on the cloth for the robot to grasp. The approach is highly effective because it depends only on geometric cues that can be identified reliably, even in the presence of changes in appearance and texture.

The robot succeeded in all 50 trials attempted on previously unseen towels with wide variations in appearance, material and size, according to the team’s report on its research, which was presented last month at the International Conference on Robotics and Automation 2010 in Anchorage. Their paper, Cloth Grasp Point Detection based on Multiple-View Geometric Cues with Application to Robotic Towel Folding, is posted online.

The system was implemented on a prototype version of the PR2, a mobile robotic platform developed by Willow Garage, using the open-source Robot Operating System (ROS) software framework. Two undergraduates, Marco Cusumano-Towner, a junior in EECS, and Jinna Lei, a senior math major, assisted on the project.

Located in Menlo Park, California, Willow Garage develops open source software and hardware to further robotics applications. Just last month, the lab awarded several research centers a prototype robot on two-year loan for developing and testing new applications. Out of a total of 78 submissions, UC Berkeley will be one of 11 institutions to receive a Willow Garage robot on loan. Moving beyond the laundry-folding application, the Berkeley team plans to next tackle the challenge of actually doing laundry, from dirty piles to neatly folded clothes, as well as other tasks like hierarchical planning, object recognition and furniture assembly.

Maitin-Shepard’s research focuses on artificial intelligence, computer vision and machine learning. He studied computer science at Carnegie Mellon University and earned a bachelor’s degree in 2008 before coming to Berkeley.

Abbeel focuses on robotics, machine learning and control. He earned his doctorate in computer science at Stanford and joined Berkeley’s EECS faculty in 2008. As part of his doctoral work, Abbeel and collaborators developed machine-learning algorithms that enable helicopters to learn to fly by watching an expert pilot fly—resulting in the most advanced autonomous helicopter aerobatics to date.