2.8.2. Hierarchical Touch Sensation Integration to Detect Complex Multi-Textured Surfaces

The third goal of this paper was to simultaneously use the LMS signals from the four fingertips of the prosthetic hand to recognize complex surfaces comprised of multiple textures. The hierarchical approach relied first upon successful texture detection localized to individual fingertips. This textural information from the four individual fingertips was integrated together to produce a higher state of knowledge at the level of the whole hand regarding the spatial layout of the multi-textured surfaces. To achieve this third goal, ten different multi-textured surfaces (S₁–S₁₀) were 3D printed using permutations of the four textures randomly generated by the MATLAB randperm function (Figure S5). An example of a complex surface comprised of four different textures is shown in Figure 7a–d. For each of the ten surfaces, 20 trials were collected to test the ability of the machine learning algorithms to distinguish between the ten different complex surfaces comprised of randomly generated permutations of four different textures. A MATLAB program was written to use the detected texture at each of the four fingertips to predict the spatial layout of the multi-textured surface that was contacted. This prediction was compared to the known database of ten surfaces comprised of multiple textures. The percentage of correct predictions was used to establish a success rate metric to quantify the capability to distinguish between the multi-textured surfaces. The speed of slip for these experiments was 20 mm/s.

(a–d) The prosthetic hand with four LMSs slid while in contact with the multi-textured surface. (e) Illustrative data from the little finger LMS showed different responses when sliding on texture 3 at 20 mm/s, (f) 60 mm/s, and (g) 100 mm/s. (h) Corresponding spectrograms showed increasing power concentrations in higher frequency bands as the sliding speed increased from 20 mm/s to (i) 60 mm/s and (j) 100 mm/s. (k) Representative time domain LMS signals from the middle finger showed different activation patterns as it slid at 20 mm/s on texture 1 (l) texture 2, (m) texture 3, and (n) texture 4. (o) Corresponding spectrogram features revealed different frequency-domain signatures specific to texture 1, (p) texture 2, (q) texture 3, and (r) texture 4.