Developing a crowd-friendly robotic wheelchair

Shoppers at Lausanne’s weekly outdoor market may have come across one of EPFL’s inventions in the past few weeks – a newfangled device that’s part wheelchair, part robot. It’s being used by researchers at EPFL’s Learning Algorithms and Systems Laboratory (LASA) to test technology they’re developing under CrowdBot, a project led by INRIA and involving a consortium of seven research organizations, including EPFL.

The project has received funding from the EU’s Horizon 2020 program in the Information and Communication Technology (ICT) section. CrowdBot aims to test the technical and ethical feasibility of having robots move through crowded areas. These robots could be humanoids, service robots or assistive robots. “You hear a lot about self-driving cars, but not about robots that could be moving around among pedestrians,” says Aude Billard, the head of LASA. “However, robotics technology is clearly going in that direction, so we have to start thinking now about all that will imply.”

“You hear a lot about self-driving cars, but not about robots that could be moving around among pedestrians” - Aude Billard, head of LASA © Alain Herzog / 2021 EPFL

Many possible scenarios

Among the various issues being studied, the most obvious concerns the safety of robot users and the people near them. LASA researchers found that existing legislation doesn’t address this, and began looking at all the possible risks, including the risk of a collision with a human.

The scientists selected a robot called Qolo – short for Quality of Life with Locomotion – to conduct their risk assessment. Qolo was originally developed at the University of Tsukuba in Japan and is intended to serve as a standing wheelchair for people with disabilities. It contains two motorized wheels and a passive exoskeleton that lets the wearer move easily from a seated to a standing position.

The LASA team conducted crash tests of their Qolo in Bern. “We ran the tests with two kinds of dummies, since the impact of a collision can vary depending on how tall the person is,” says Diego Paez, a postdoc at LASA. “In children for example, the most vulnerable area is the head, whereas in pregnant women it’s the abdomen.” The researchers discovered that collisions at even low robot speeds, like under 6 km/h, can cause serious injuries. That makes preventing these collisions even more important.

An active navigation system

The first step was to modify Qolo so that it can analyze and react to its surroundings. The scientists equipped the robot with various sensors including cameras on the front and a Lidar system with lasers on the front and back. “It’s important for the robot to have a 360° view of its surroundings so that it can avoid obstacles in front of it and behind it. It also needs to know what’s behind it in case it has to back up quickly to avoid a collision,” says Paez. “The Lidar system detects all kinds of obstacles, and the cameras let the robot know whether the obstacles are pedestrians.”

The team also installed bumpers on Qolo’s front. “The bumpers tell the robot that it’s come into contact with something and measure the contact force, so that the maximum force can be held very low while the robot still moves,” says Paez. In other words, Qolo isn’t programmed to stop if it hits an obstacle but rather to move around it. “An abrupt halt in the middle of a crowd could be even more dangerous for people close to the robot,” he says.

The data from Qolo’s sensors are combined with people detection and tracking algorithms to estimate how many people are around the robot and what directions they’re moving in. LASA researchers developed a special navigation algorithm for Qolo that lets it identify the best path to take in just a few milliseconds, meaning it can respond quickly in packed crowds.

Predicting the unpredictable

Despite the engineers’ technological prowess, their robot can’t (yet) predict sudden movements like swift changes in direction. “We can’t really simulate what people will do different situations because everyone reacts differently. That’s why we need to test Qolo in real-world conditions,” says Paez. Hence the trial runs in Lausanne’s open market.

Diego Paez, post-doctoral researcher at LASA, tests the Qolo robotic wheelchair in the heart of Lausanne © Alain Herzog / 2021 EPFL

There the engineers can get valuable feedback on all the robot’s systems – from its hardware to its algorithms – and on the user experience. The initial results are promising; pedestrians seem to behave normally around the machine, which is a big plus for data collection. “We still need to analyze the data, but it appears that the semi-autonomous dimension of the robot works well,” says Paez. Billard adds: “Users guide Qolo by moving their torso to indicate what direction to go in. If an obstacle suddenly appears, the robot will respond immediately to avoid it. That kind of assisted navigation can be a real advantage for disabled persons.”

Risk awareness

With advancements in robotics taking place at a rapid pace, we could start seeing more and more such devices on our roads and sidewalks – like delivery robots, for example. The LASA team nevertheless stresses one crucial point: it will be essential to develop effective ways for minimizing the probability of collisions and other accidents. “Crash tests have shown that the risk of injury could be high and sometimes exceeds what’s permissible for automobiles,” says Paez.

“Now we need to work on a control system for mitigating this risk, whether by lowering the robot’s speed or improving its shock absorption capacity,” says Billard. “And it’s crucial for these findings to be taken into account in future legislation. These laws could include setting a speed limit for assistive robots like Qolo or restricting the ability of some kinds of vehicles, like delivery robots, to operate in highly frequented areas.”