Meta develops multi-functional tactile skin Reskin, which can quickly improve machine sensitivity and tactile sensing accuracy.
Nowadays, artificial intelligence gradually integrates with human senses such as sound and vision, making communication between people more convenient. However, it is still challenging to integrate artificial intelligence with human touch.
To solve this problem, Meta AI and Carnegie Mellon University successfully developed a multifunctional, replaceable and durable tactile skin, and named it ReSkin. It can quickly improve the tactile sensing accuracy and sensitivity of the machine in the application process.
(Source:Proceedings of Machine Learning Research )
Related papers are titled "Multifunctional, Replaceable and Lasting Touch Skin" (ReSkin：versatile，replaceable，lasting tactile skins) published in [1].
Matai visiting researcher Raunaq Buhanlan (Raunaq Bhirangi), Tess Hellebrex, postdoctoral fellow and scientist of Pittsburgh Yuan Artificial Intelligence Research Center (Tess Hellebrekers), Mel Majdi, Professor of Robotics Institute of Carnegie Mellon University (Carmel Majidi), Abbina Gupta, Associate Professor of Robotics Institute of Carnegie Mellon University (Abhinav Gupta) is the author of the paper.
It is mentioned in the paper that ReSkin relies on machine learning technology and magnetic sensing technology, and has the advantages of being cheap, multifunctional, durable and replaceable. Specifically, ReSkin has low production cost, only 2-3mm thick, and can interact with the machine model more than 50,000 times. In addition, it has a high spatial-temporal resolution with an accuracy of 90%.
Figure | ReSkin is the size of a coin and easy to manufacture (Source:PMLR)
Thin and high-precision specifications make it suitable for all kinds of machines, such as robot hands, tactile gloves, arm sleeves, etc. For specific processes, ReSkin can also provide tactile signals (high frequency) for the sliding, throwing, catching and clapping operations of the machine.
When Reskin is applied to different products, a large amount of relevant data will be generated, which can help researchers improve their tactile perception ability in AI systems.
Figure | Frequency changes of sensors applying ReSkin under different magnetic fields (Source:Proceedings of Machine Learning Research)
For example, ReSkin is an elastic sheet that can change its shape and contains magnetic particles. When its shape changes, it will release different magnetic signals. Researchers can use magnetometer to measure these changes, and use data-driven technology to convert the measured data into information such as contact position and applied force.
At present, many tactile sensing experiments are limited to one sensor level. This is because every time ReSkin is replaced, the machine needs to build a new model, which reduces the transmission efficiency.
Moreover, in different scenarios, each sensor needs to be thoroughly calibrated with the initial calibration procedure to match its individual response. This means that the calibration procedure must also adapt to these changes. In addition, as time goes by, soft skin like ReSkin will deform and need to be replaced, so it is difficult to popularize and apply it to different interactive scenarios.
In order to simplify the replacement process of Reskin, the researchers made innovations in three aspects.
Firstly, the researcher separates the internal circuit of the sensor from the passive interface, and does not need to be electrically connected with the traditional measuring electronic equipment. This operation effectively improves the sensitivity of the sensor, and it is as simple as sticking a sticker when replacing worn ReSkin.
Secondly, the researchers use the output data of several sensors to improve the model mapping. Through this operation, researchers can use a higher data diversity training model. This helps the sensor to output more effective and generalized data.
Thirdly, thanks to the self-supervised learning mode of the machine, researchers don’t have to collect calibration data for each new sensor, but use a small amount of unlabeled data to automatically fine-tune the sensor.
It is known that the existing camera-based tactile sensor requires a very small contact distance between the surface and the camera, which leads to a heavier machine. In contrast, ReSkin can cover the hands and arms of humans and robots, which facilitates researchers to develop multifunctional, expandable and inexpensive tactile modules. This is impossible for the existing artificial intelligence tactile system.
In order to highlight the practical value of ReSkin and show its unique charm of promoting the development of artificial intelligence, researchers apply it to different machine scenes. From grasping tiny objects to measuring the force exerted by the dog’s feet, from building a continuous ReSkin with wide coverage to measuring the field contact force, ReSkin has shown its extremely high flexibility and practicality.
Although researchers have demonstrated the technical advantages of ReSkin in contact location and force prediction, ReSkin still has great development potential in the future.
The experiment of this paper is based on the single-point contact of the machine, and the goal of scientists is to further study the application of ReSkin under multi-point contact. Another interesting future development direction is to specifically analyze the influence of external magnetic field and metal objects on ReSkin’s perception ability.
In addition, based on ReSkin’s high time resolution of 400Hz, researchers can make use of this advantage and use dynamic time series data to create better machine models. In a word, scientists believe that ReSkin will promote the machine’s touch perception ability and be able to apply it to practice.
