The BionicMobileAssistant moves autonomously in three dimensions and can independently detect objects, adapt its grip, and work together with people. The information is processed by a neural network that has been trained in advance with the aid of data augmentation.
In the future, operators and robots will work ever more closely together. That is why here at Festo, we are working intensively on systems which, for example, can relieve people of monotonous or hazardous activities while at the same time posing no risk. Artificial intelligence plays a central role in this regard.
The BionicMobileAssistant, which was developed in collaboration with ETH Zurich, consists of three subsystems: a mobile robot, an electric robot arm, and the BionicSoftHand 2.0. The pneumatic gripper is inspired by the human hand and is an advanced version of the BionicSoftHand developed in 2019.
The DynaArm, an electric robot arm, can carry out fast and dynamic movements thanks to its lightweight design with highly integrated actuator modules that weigh just one kilo. These modules, known as DynaDrives, pack the motor, gear unit, motor control electronics, and sensors into a very compact design. In addition, the arm features an extremely high power density that, with a kW at 60 Nm drive torque, far exceeds that of conventional industrial robots.
Thanks to model-based force control and control algorithms to compensate for dynamic effects, the arm is able to respond well to external influences and therefore interact extremely precisely with its environment. It is controlled by a Ballbot via an EtherCAT communication bus. Thanks to its modular design, the DynaArm can quickly be brought into operation and easily maintained.
The Ballbot is based on a sophisticated drive concept – it balances on a ball powered by three Omniwheels. This means that the BionicMobileAssistant can move freely in every direction. The robot only touches the floor at one point at a time and can therefore navigate through tight spaces. In order to keep its balance, it must stay moving. The movements are planned and coordinated using planning and control algorithms that are saved on a powerful computer in the body of the Ballbot.
The robot’s stability is achieved entirely dynamically – in the event of external influences, the Ballbot can quickly rotate its ball in order to maintain its balance. Using an inertial measuring unit and position encoders on the wheels, it tracks its movements and the relative tilt of the system. Based on this data, an optimization program calculates how the robot and arm must move in order to bring the hand into the target position and simultaneously stabilize the robot.
The fingers of the pneumatic robot consist of flexible bellows structures with air chambers, surrounded by a firm yet yielding knitted fabric. This makes the hand light, adaptive, and sensitive, yet capable of exerting strong forces. Similar to the BionicSoftHand released in 2019, the pneumatic fingers are actuated by a compact valve manifold with piezo valves mounted directly on the hand.
The hand wears a glove with tactile force sensors on the fingertips, the palm, and the outer sides of the robot hand. This allows it to sense how hard the item to be gripped is and how well it is positioned in the hand, and to adapt its gripping force to the particular item – just as people do. There is also a depth camera on the inside of the wrist for visual object detection.
Using the camera images, the robot hand can detect and grip a range of objects, even if they are partially obscured. Once the hand has been correctly trained, it can use the collected data to assess the objects and distinguish good from bad, for example. The information is processed by the neural network that was previously trained using data augmentation.
In order to achieve the best possible results, the neural network requires a great deal of information that it can use to train itself. This means the more training images available, the more reliable it becomes. Since this is usually a time-consuming process, it makes sense to increase the size of the data set automatically.
This process is known as data augmentation. By slightly modifying a few initial images – such as by changing the background, lighting conditions, or viewing angles – and then duplicating them, the system can create an extensive data set that it can work with autonomously.
The system has its entire power supply on board – the battery for the arm and robot is located in the body. The cartridge for the pneumatic hand’s compressed air supply is integrated into the upper arm. This not only makes the robot mobile, but also enables it to move autonomously.
The algorithms saved on the main computer also control the system’s autonomous movements. They anticipate how the arm and the ball will have to move to reach certain target points while maintaining the robot’s balance. With the help of two cameras, the robot autonomously orients itself in three dimensions – one camera scans for predefined fixed points in the environment in order to position itself absolutely, while a second camera uses the ceiling structure to anticipate movement.
Its mobility and autonomous power supply allow the BionicMobileAssistant to be used flexibly for different tasks at varying locations – in line with the concept of continuously changing production processes.
The system is perfect for use as a direct assistant to humans, for example as a service robot, as a helping hand in assembly operations, or to assist workers in performing ergonomically strenuous or monotonous tasks. It could also be used in environments where people cannot work due to hazards or restricted accessibility, for example.
Thanks to its modular design, the BionicSoftHand 2.0 can also be quickly mounted to other robot arms and brought into operation. In combination with the BionicCobot or the BionicSoftArm, for example, the gripper forms a completely pneumatic robot system that can work hand in hand with humans due to its inherent ability to give way.