Mobile robot system with pneumatic gripping hand  

The BionicMobileAssistant moves autonomously in space and can independently recognise objects, grasp them adaptively and work on them together with humans. The processing of the acquired information is performed by a neural network that has been trained in advance using data augmentation.


In the future, workers and robots will work together more and more closely. For this reason, we at Festo have been looking intensively into systems that could, for example, relieve people of monotonous or dangerous activities and at the same time pose no risk. Artificial intelligence plays a central role here.

Modular assistance system

In cooperation with ETH Zurich, the BionicMobileAssistant was developed, which consists of three subsystems: a mobile robot, an electric robot arm and the BionicSoftHand 2.0. The pneumatic gripper is inspired by the human hand and is a further development of the BionicSoftHand from 2019.

DynaArm: dynamic robot arm with high power density

With the DynaArm, an electric robot arm, fast and dynamic movements are possible. This is ensured by its lightweight design with highly integrated drive modules weighing only one kilogram. In these so-called DynaDrives, the motor, gear unit, motor control electronics and sensors are installed in a very small space. In addition, the arm has a high power density which, with 1 kW at 60 Nm drive torque, far exceeds that of conventional industrial robots.

Thanks to model-based force control and control algorithms to compensate for dynamic effects, the arm can react well to external influences and thus interact very sensitively with its environment. It is controlled by the ballbot via an EtherCAT communication bus. Thanks to its modular design, the DynaArm can be quickly put into operation and easily maintained.

Ballbot: mobile robot application with special drive

The ballbot is based on a sophisticated drive concept: it balances on a ball driven by three omniwheels. This allows the BionicMobileAssistant to manoeuvre in any direction. The robot only touches the ground at one point at a time and can therefore navigate through narrow passages. In order to maintain its balance, it must move continuously. The planning and coordination of the movements are carried out using planning and control algorithms that are stored on a powerful computer in the body of the ballbot.

The stability of the robot is realized purely dynamically - in case of external influences, the ballbot can quickly set the ball in rotation and thus keep its balance. Using an inertial measuring unit and position encoders on the wheels, it records its movements and the relative inclination of the system. Based on this data, an optimisation program calculates how the robot and arm must move to bring the hand into the target position and stabilise the robot at the same time.

The fingers of the pneumatic robot hand consist of flexible bellows structures with air chambers, covered by a firm and at the same time pliable textile knit. This makes the hand light, flexible, adaptable and sensitive, yet capable of exerting strong forces. As with the BionicSoftHand from 2019, the pneumatic fingers are also controlled via a compact valve terminal with piezo valves, which is mounted directly on the hand.

Finely tuned gripper with fingertip sensitivity

The hand wears a glove with tactile force sensors on the fingertips, the palm and the outside of the robot hand. This allows them to feel how hard the object to be gripped is and how well it fits in the hand, and adapt their gripping force to the object in question – just like we humans do. In addition, a depth camera is located on the inside of the wrist for visual object detection.

Click on the image to open the graphic with label in a new window.
Click on the image to open the graphic with label in a new window.

Object detection by means of a neural network

With the help of the camera images, the robot hand can recognize and grip various objects, even if they are partially covered. After appropriate training, the hand can also assess the objects on the basis of the recorded data and thus distinguish good from bad, for example. The information is processed by the neural network, which was trained in advance using data augmentation.

Extensive data sets through data augmentation

In order to achieve the best possible results, the neural network needs a lot of information with which it can orient itself. This means the more training images are available to it, the more reliable it becomes. Since this is usually time-consuming, automatic augmentation of the database is a good idea.

This procedure is called data augmentation. By marginally modifying a few source images – for example, with different backgrounds, lighting conditions or viewing angles – and duplicating them, the system obtains a comprehensive data set with which it can work independently.

  • BionicMobileAssistant

    Reliable assistance system: ballbot, DynaArm and BionicSoftHand 2.0 in action

  • BionicMobileAssistant

    Visual object recognition: a depth camera on the wrist

  • BionicMobileAssistant

    Digital learning: data augmentation for training the neural network

  • BionicMobileAssistant

    Reliable manoeuvres: even if pushed, the ballbot will balance and not fall over.

  • BionicMobileAssistant

    Optimum traction: the three Omniwheels are also driven by one DynaDrive each.

  • BionicMobileAssistant

    Autonomous navigation: orientation in space with the help of a second camera

  • BionicMobileAssistant

    Mobile compressed air supply: the cartridge for the pneumatic hand is integrated into the upper arm

  • BionicMobileAssistant

    Modular concept: the BionicSoftHand 2.0 on the BionicCobot

Mobile use at changing locations

The system has its entire power supply on board: the battery for the arm and robot sits inside the body. The compressed air cartridge for the pneumatic hand is installed in the upper arm. This means that the robot is not only mobile, it can also move autonomously.

The algorithms stored on the master computer also control the autonomous movements of the system. With a view to the future, they plan how the arm and the ball must move in order to reach certain target points while maintaining balance. With the help of two cameras, the robot orients itself independently in space: one camera searches for predefined fixed points in the environment to position itself autonomously, while a second camera uses the ceiling structure to estimate movement.

Its mobility and autonomous energy supply enable the BionicMobileAssistant to be used flexibly for different tasks at changing locations – in line with the constantly changing production environment.

Versatile application possibilities

The system would be predestined for use as a direct assistant to humans, for example as a service robot, as a helping hand in assembly or to support workers in ergonomically stressful or monotonous work. It could also be used in environments where people cannot work, for example due to hazards or limited accessibility.

Hand in hand with humans

Thanks to its modular concept, the BionicSoftHand 2.0 can also be quickly mounted and commissioned on other robot arms. Combined with the BionicCobot or the BionicSoftArm, the gripper forms, for example, a completely pneumatic robot system that can work hand in hand with humans due to its inherent flexibility.