ABSTRACT
Conventional prosthetic hands require users to activate designated muscles or press buttons to select among predefined grasping patterns. These methods are time-consuming and increase muscle fatigue. This study proposes a regression model that differentiates multiple muscle activation patterns allowing the user to select a desired grasping pattern. We classified four hand primitives and three force intensities, which can reflect the intention of prosthetic hand users. An 8-channel band-type sEMG sensor was used to measure myoelectric signals from an amputated upper-arm. To acquire the sEMG data, the amputee was instructed to imagine four hand primitives (fist, open hand, flexion, and extension) with three levels of force intensity (low, medium, and high). Time-domain features (mean average value, variance, waveform length, and root mean square) were extracted from the sEMG signal and classified using a Support Vector Machine. The hand primitives and force intensities had accuracies of 95% and 90%, respectively. Results indicate the regression model reflected the user’s intention to select different grasping patterns, and is thus expected to improve the quality of life of amputees.
-
KEYWORDS: Prosthetic hand, Grasping, Regression model, Movement primitive, Electromyographic signals
-
KEYWORDS: 전동 의수, 잡기 동작, 추정 모델, 동작 프리미티브, 근전도 신호
1. Introduction
A prosthetic hand is an artificial device which can replace a missing upper-limb that is lost due to diseases, accidents, or congenital conditions. It mainly helps amputees grasp objects and increase their QoL (Quality of Life).
1 Aesthetic, body-powered, and myoelectric prosthetic hands are most widely used. Among them, the myoelectric type is the most expensive since it provides various automatically controlled motions.
Amputees are expected to increase up to 3.6 million by 2050 worldwide.
2 However, despite the large population, the usage rate of myoelectric prosthetic hand is extremely low. 70% of upper-limb amputees are using prosthetic hands, but less than 5% are using myoelectric hands.
3 It is due to the low usability of conventional prosthetic hands. Most of them use 2-channel EMG sensors and buttons to select grip modes.
4 Michelangelo Hand requires a user to activate extensor or flexor to grasp or ungrasp an object, and thumb-opposition is switched by co-contraction. Bebionic and i-Limb requires users to activate extensor or flexor repeatedly or press buttons to select grip modes. These conventional interfaces, which utilize repeated activation of designated muscles, not only increase muscle fatigue but also limit the user to perform tasks that require both hands.
To enhance the usage rate of the prosthetic hand and its usability, an interface that doesn’t require both hands or repeated muscle activation to select grip modes is necessary. Therefore, a new reliable muscle activation process is needed. In this paper, grasping tasks are viewed in three different steps; extending, reaching, and grasping. A novel method to select a multiple signal with a simple manner is proposed. This method uses eight sEMG sensors on the upper limb to make a regression model of hand primitives and force intensities. Each regression model is used for the classification required in the grasping motion sequence specified above. Briefly, in the extension phase, the hand primitive regression model is used to determine the grasping motion of the amputee. Then a force intensity regression model is used in the reaching phase to determine how strong the grasping motion is. Therefore, by applying the regression model, which is obtained by experiments, to the prosthetic hand, it can reflect the user’s intention easily and accurately for the grasping activities.
2. Materials & Methodologies
2.1 Data Acquisition & Data Processing
2.1.1 Electromyography (EMG) Sensor
In human movement, muscle is the actuator, which means exercise occurs due to muscle contraction. EMG sensor is a sensor that measures the amount of muscle contraction, so that human motion and intention can be estimated effectively.
In this study, a sEMG sensor is used to express hand primitives and force intensities for upper-limb amputee. sEMG is a useful and effective signal for estimating the muscle activation pattern and force intensity from the upper-limb residual.
Previous studies that used sEMG have indicated that sEMG sensors that requires attachment of electrodes are cumbersome and difficult to find the exact position for the electrodes attachment. Therefore, a sEMG sensor that is easy to calibrate and wear should be used. An 8-channel band-type sEMG sensor is used to enhance usability and feasibility. It has eight sEMG sensors combined by elastic bands, which makes it easy to wear and calibrate because the sEMG sensors are attachable all around the arm.
2.1.2 EMG Signal Processing
The raw sEMG signal measured from the 8-channel band-type sEMG sensor is filtered to eliminate high frequency noise involved within the signal. The transfer function of the low-pass filter shown below is otherwise called quasi-tension filter since the filtered data show a high correlation to actual muscle tension, and therefore, it is appropriate to represent muscle activity.
5
The magnitude and frequency properties of sEMG signal and quasi-tension have been known to be highly sensitive and variable with many factors such as the skin condition and the electrode attachment position. To avoid the effect of these problems, EMG signal must be normalized. The most popular normalization method of EMG signal is MVC (Maximum Voluntary Contraction). For normal people, MVC is found by exerting the maximum force when restraining the arm movement. However, for amputees, this method is impossible to apply. Also, the amplitude of the quasi-tension of the amputees is small compared to the normal people due to the muscle degeneration, which results low resolution in low-level activation. Therefore, MVC is selected as 70% of the maximum signal amplitude when maximum force is applied when performing flexion, extension, and co-contraction.
2.2 Data Segmentation
The sEMG signal should be measured continuously to operate the prosthetic hands. However, not all the acquired sEMG signal is used. Only the signal obtained when the user wants the prosthetic hand to operate should be used. Therefore, an appropriate sEMG signal onset criterion is necessary. For this, establishing threshold is the most generally used method.
Fig. 1Double-threshold method for data segmentation
There are many ways to set the sEMG signal onset threshold. The simplest method is to use single-threshold method. However, the single-threshold method has a high degree of sensitivity because it has only one degree of freedom.
6 Therefore, double-threshold method is used to solve this drawback.
7 The principle of the double-threshold method is as follows: sEMG signal is measured only when it exceeds the first threshold, like the single-threshold method. If the number of consecutive samples or length of time of the measured sEMG signal exceeds the second threshold, the very sample or time when the signal first exceeds the first threshold is set as the beginning of the activation time. The general formula of the double-threshold method is as follows:
Out of diverse classification elements, feature set has been shown to have the highest classification accuracy and can be applied in a more various way than other elements.
8 Features are extracted from the processed sEMG for classification. Finding the most appropriate features is important to increase the classification accuracy.
9 There are diverse feature sets, such as Hudgins’ time-domain features (TD), spectral power magnitudes (SPMs), and short-time Fourier transform (STFT), but in this study TD feature set is chosen for the following reasons: TD feature set is simple to implement and has been shown to be effective for classification when the arm is moving.
10 The most commonly used features of TD feature set are shown in
Table 1.
Table 1Definitions for time domain features4,11
Table 1
|
Feature |
Formula |
|
Mean absolute value (MAV) |
1N∑i=1Nxi
|
|
Variance (VAR) |
1N-1∑i=1Nxi2
|
|
Waveform length (WL) |
∑i=1N-1xi+1-xi
|
|
Root mean square (RMS) |
1N∑i=1Nxi2
|
Mean absolute value (MAV), variance (VAR), waveform length (WL), root mean square (RMS) indicates the mean value, variance, variation, and energy of the muscle activation, respectively.
2.4 Feature Classification through Machine Learning
Feature extracted in real time should be classified into appropriate clusters by comparing with the existing database through classifier.
Support Vector Machine (SVM) is a universal machine learning method that can classify data sets with very high accuracy.
12 The SVM classifies each cluster by placing a hyper-plane between the two data sets. It is important to select a hyper-plane that has the smallest margin from the hyper-plane to support vector, which is the data element closest to the hyper-plane. Since SVM generally classifies only two data sets, it is necessary to apply multiple SVMs to perform multi-classification.
13 We have used fivefold cross-validation to design the classifier. In this study, we have employed SVM toolbox in the MATLAB.
Finding hand primitives that amputees can perform distinctively is necessary. Diverse amputees should be able to perform the hand primitives; therefore, it is important to test multiple subjects to find them. For this, normal subjects were instructed to imagine different hand primitives while their one hand and wrist are constrained. The confusion matrix for the hand primitives when using SVM is listed below.
The above table is a confusion matrix showing the accuracy of each hand primitives that are classified. Horizontal and vertical represent each hand primitive, and diagonal column indicate the accuracy of each hand primitives that can be classified.
Hand primitives with the highest classification accuracy were fist, open-hand, flexion, and extension. Therefore, the four hand primitives were selected for the experiment in this study.
Fig. 2Confusion matrix for eight different hand primitives
3. Experiments
3.1 Experimental Setup
3.1.1 Experimental Device
In this study, ‘MYO Armband’ is used to measure the muscle activation from the subject’s upper-limb. ‘MYO Armband’ is a gesture recognizing wrist band made by Thalmic Lab. It has eight individual sEMG channels and nine degrees of freedom IM sensors. The 8-channel band-type sEMG sensor is easy to wear and has high usability, therefore, it is selected for data acquisition. The position where each channel is attached to is not directly related to the muscle for moving the wrist or finger. However, the muscle synergy effect of the muscular activation such as the Brachioradialis muscle, which is located deeply, provides the data of the intended muscle activation. First, second, and third channels are located in triceps brachii direction and sixth, seventh, and eighth channels are located in biceps brachii direction.
3.1.2 Experimental Schematic
Raw sEMG signal is acquired from the 8-channel band-type sEMG sensor, and it is filtered to obtain quasi-tension. TD feature set is then extracted from the raw sEMG signal or the quasi-tension signal. The extracted feature set is applied to find the regression model for classification of hand primitive and force intensity.
The classification accuracy increases accordance with the size of the feature set. However, the increment of the classification accuracy is small when the number of features is more than two. Therefore, using three features is expected to achieve the similar performance as using all features.
14
The purpose of this experiment is to find the regression model for hand primitive classification of the upper-limb amputee.
Fig. 3Vector space displacement for subject reinforcement
Table 2Subject information
Table 2
|
Age |
60s |
|
Gender |
Male |
|
Accident sequence |
Electric shock |
|
Accident occurred year |
2000 |
|
Affected area |
Left arm – 7 cm under the elbow
Right arm-Above the elbow |
|
Myoelectric prosthetic hand using year (term) |
2001 (3 months) |
|
Muscle suturing type |
Flexor, extensor suturing together |
The subject was seated on a chair with the elbow positioned perpendicular. sEMG signal from the amputated upper-limb using the 8-channel band-type sEMG sensor. The sEMG sensor was attached on the amputated upper-limb and calibrated to ignore the disturbance by the different sensor attachment site.
Then the acquired raw signal is normalized based on MVC. To obtain MVC, the subject was instructed to perform flexion, extension, and co-contraction with the maximum force. 70% of the acquired MVC is used for the normalization.
To find the regression model, the subject was instructed to imagine performing four different hand primitives (fist, open hand, flexion, and extension) with the amputated upper-limb. To help the subject imagine the hand primitives, pictures of the hand primitives were displayed on the monitor. The subject was instructed to rest for 3 seconds and imagine a hand primitive for another 3 seconds sequentially. These are regarded as rest session and activate session, respectively, and a rest session and an active session forms a set. To inform the subject when to rest and when to imagine the hand primitives, beep sound was given at the end of each session. To reinforce the subject, an octagonal vector-space was displayed on the monitor at the end of each set. The vector-space is made by assigning 8 channels of the sEMG sensor to vertices of the octagon. For each channel, the magnitude of the quasi-tension signal was plotted as the distance from the center of the octagon to the end of the vertex. The experiment was done 5 sets for each hand primitive.
During the process, sEMG signal was obtained for each channel and only the section which satisfied the pre-determined double-threshold criterion was used for feature extraction. Three features were extracted from each channel, which results 120 features after 5 sets. The features were used to build the classifier.
3.4 Experiment for Force Intensity Classification
The purpose of this experiment was to find the regression model for force intensity from the amputated upper-limb when exerting co-contraction. This can be applied to the prosthetic hand to select grasping strength of the prosthesis.
Force intensity levels were selected based on the research about the required grip forces to perform daily tasks, which are listed below.
15
The experiment was performed in the same process of the experiment for hand primitive classification. The only difference was the instruction the subject was given; instead of imaging certain hand primitive, the subject was instructed to image exerting co-contraction with certain force intensity. To train the subject three different force intensities, three different weights were placed on the prosthetic hand the subject has used on a daily basis, and the subject was asked to remember the force intensity formed in the amputated upper-limb.
Table 3Daily tasks and weight of the necessary object
Table 3
|
Force step |
Daily tasks (Object) |
Approximate
force required |
|
Low |
Brush teeth (Toothbrush) |
0.98 N |
|
Medium |
Drink water (Cup with water) |
4.90 N |
|
High |
Hold bottle (2 L water bottle) |
19.60 N |
Fig. 4Experimental schematic for hand primitive classification
Fig. 5Vector space for MAV, VAR, RMS, and WL features when the amputee performs different hand primitives and force intensities
The experiment was performed for 4 sets, each set consisted of a rest session and an activation session.
Features were extracted from the quasi-tension that satisfied the double-threshold criterion. Three features were extracted for each channel, which results 96 features after 4 sets. The features were used to build the classifier.
3.5 Experiment for Regression Model Evaluation
To evaluate the classification accuracy of the designed regression model, the subject was instructed to perform four hand primitives and three force intensities for 5 sets and 4 sets, respectively. Each set was done for 2 seconds. The acquired data was classified by using the regression model made from the above experiments, and the classification accuracy was evaluated.
Through this, a regression model was designed, which can enhance the usability of the prosthetic hand by enabling the user to select grip modes and grasping forces with a more intuitive interface and higher classification accuracy.
4. Result and Discussion
4.1 Threshold
It is necessary to set an appropriate double threshold to select the criteria of EMG signal onset. For this purpose, the quasi-tension was measured while resting comfortably and imaging certain grip motions, with the EMG sensor attached on the upper arm of the subject.
Table 4 shows the resultant quasi-tensions for 5 sets.
Table 4Mean values of quasi-tension for different grip motions
Table 4
|
Set No. |
Motion1 |
Motion2 |
Motion3 |
Motion4 |
Rest |
|
1 |
107.73 |
67.83 |
174.88 |
197.79 |
3.77 |
|
2 |
190.00 |
103.05 |
135.72 |
180.54 |
35.67 |
|
3 |
123.14 |
107.59 |
165.83 |
175.48 |
8.24 |
|
4 |
228.84 |
68.76 |
221.22 |
130.84 |
10.12 |
|
5 |
129.10 |
104.76 |
127.43 |
163.75 |
3.75 |
|
Avg. |
155.76 |
90.40 |
165.02 |
169.68 |
12.31 |
According to the results in
Table 4, the average of the quasi-tension for the four different motions are 145.22% and 12.31% when at rest. Therefore, the first onset threshold
h1 is set to 78.77%, which is the intermediate value between 145.22% and 12.31%.
The time required to reach the object for grasping is approximately 450 ms.
16 Since the reaching time of prosthetic hand users is longer than that of normal people, the second threshold
h2 is acceptable to be set to 450 ms.
The raw EMG data and the quasi-tension measured when the subject was instructed to imagine 4 different grip motions are shown in
Fig. 6. Root Mean Square (RMS) was extracted from the raw EMG data for each of the 8 channels, and Mean Absolute Value (MAV), Variance (VAR), and Waveform Length (WL) were extracted from the quasi-tension for each of the 8 channels, which makes 32 features in total. Three features are selected for classification training, and to find the feature combination with the highest accuracy, the classification accuracy for different combinations were tested. SVM classifier was built with MATLAB classification toolbox. The classification accuracies for different feature combinations are listed in
Table 5. Consequently, the combination of using MAV, VAR, and RMS had the highest accuracy of 95%, and therefore it is considered to be the most efficient feature combination.
Table 5Classification accuracy for hand primitive and force intensity classification
Table 5
Features used
for each channel |
Classification accuracy (%) |
|
Hand primitive |
Force intensity |
|
MAV, VAR, WL |
90 |
70 |
|
MAV, VAR, RMS |
95 |
80 |
|
MAV, WL, RMS |
85 |
90 |
4.3 Force Intensity Experiment Result
The raw EMG data and quasi-tension were measured when the subject was instructed to apply 3 stepwise force intensities. As in the Experiment 1 result, RMS was extracted from the raw EMG data for each of the 8 channels, and MAV, VAR, and WL were extracted from the quasi-tension for each of the 8 channels. Three features are selected for classification training, and the classification accuracy for different combinations of features are listed in Table 6. Consequently, the combination of using MAV, WL, and RMS had the highest accuracy of 90%, and therefore it is considered to be the most efficient feature combination.
In conclusion, it is possible to implement the interface that reflects the user-intended grip motion and force intensity with high fidelity by using machine learning of different features of EMG signal.
5. Conclusion
We designed a regression model that can classify four hand primitives and three force intensities based on sEMG from the amputated upper-limb. By implementing the model to the prosthetic hand, it was possible to reflect the intention of the user.
To achieve this, sEMG was signal-processed to gain the quasi-tension, which is closely related to the muscle tension. The data was normalized by 70% MVC. TD feature set (MAV, VAR, WL, and RMS) was extracted from the raw sEMG signal or the quasi-tension that exceeded the pre-determined double-threshold. The extracted features were classified by using SVM.
Two experiments were performed to design regression models for hand primitive and force intensity classification. For hand primitive classification, the subject was instructed to imagine performing four different hand primitives (fist, open hand, flexion, and extension) using the amputated upper-arm. For force intensity classification, the subject was asked to perform co-contraction when exerting three different levels of force intensity.
As a result, four different hand primitives were classified with high accuracy of 95% when using MAV, VAR, and RMS feature set. Three levels of force intensity were classified with high accuracy of 90% when using MAV, WL, and RMS feature set. By using the regression model obtained from the experiments, we were able to classify hand primitives and force intensities in real-time. The classified result is used for the prosthetic hand operation, and it confirmed satisfactory response from the amputee. Therefore, by applying the regression model to the prosthetic hand, it can transmit the user’s intention easily and accurately, which will consequently enhance the QoL of amputees.
To improve our interface reflect user’s controlling intention better, further works should be done. This study was performed when the user is not moving, which means the disturbance that could occur due to the translational movement of the hand wearing the prosthetic hand was ignored. However, to perform daily tasks, being able to select grip modes when the hand is reaching to an object is essential. Therefore, for the next study, we propose improving the designed regression model, so it can classify different hand primitives and force intensities in the reach-to-grasp phase.
ACKNOWLEDGMENTS
This paper was presented at ISGMA 2017
This paper is supported by CESEE (Convergence Education for Sustainable Energy and Environment). Thank you to the Pohang General Welfare Center for the Disabled and the participant who participated in the experiment.
Appendix
- APPENDIX
Fig. 6Raw EMG data and quasi-tension signal for 4 grip motions (blue-raw EMG data, red-filtered quasi-tension data)
NOMENCLATURE
Time when signal change is detected
Time when onset is detected
REFERENCES
- 1.
Luchetti, M., Verni, G., Sacchetti, R., and Rossi, N., “Impact of Michelangelo Prosthetic Hand: Findings from a Crossover Longitudinal Study,” Journal of Rehabilitation Research and Development, Vol. 52, No. 5, p. 605, 2015.
10.1682/JRRD.2014.11.0283
- 2.
Ziegler-Graham, K., MacKenzie, E. J., Ephraim, P. L., Travison, T. G., and Brookmeyer, R., “Estimating the Prevalence of Limb Loss in the United States: 2005 to 2050,” Archives of Physical Medicine and Rehabilitation, Vol. 89, No. 3, pp. 422-429, 2008.
10.1016/j.apmr.2007.11.005
- 3.
McFarland, L. V., Winkler, S. L. H., Heinemann, A. W., Jones, M., and Esquenazi, A., “Unilateral Upper-Limb Loss: Satisfaction and Prosthetic-Device Use in Veterans and Servicemembers from Vietnam and OIF/OEF Conflicts,” Journal of Rehabilitation Research and Development, Vol. 47, No. 4, p. 299, 2010.
10.1682/JRRD.2009.03.0027
- 4.
- 5.
Clancy, E. A., Bida, O., and Rancourt, D., “Influence of Advanced Electromyogram (EMG) Amplitude Processors on EMG-to-Torque Estimation during Constant-Posture, Force-Varying Contractions,” Journal of Biomechanics, Vol. 39, No. 14, pp. 2690-2698, 2006.
10.1016/j.jbiomech.2005.08.007
- 6.
Micera, S., Vannozzi, G., Sabatini, A., and Dario, P., “Improving Detection of Muscle Activation Intervals,” IEEE Engineering in Medicine and Biology Magazine, Vol. 20, No. 6, pp. 38-46, 2001.
10.1109/51.982274
- 7.
Bonato, P., D'Alessio, T., and Knaflitz, M., “A Statistical Method for the Measurement of Muscle Activation Intervals from Surface Myoelectric Signal During Gait,” IEEE Transactions on Biomedical Engineering, Vol. 45, No. 3, pp. 287-299, 1998.
10.1109/10.661154
- 8.
- 9.
Keating, J., “Relating Forearm Muscle Electrical Activity to Finger Forces,” Worcester Polytechnic Institute, 2014.
- 10.
Englehart, K., Hudgin, B., and Parker, P. A., “A Wavelet-Based Continuous Classification Scheme for Multifunction Myoelectric Control,” IEEE Transactions on Biomedical Engineering, Vol. 48, No. 3, pp. 302-311, 2001.
10.1109/10.914793
- 11.
Phinyomark, A., Hirunviriya, S., Limsakul, C., and Phukpattaranont, P., “Evaluation of EMG Feature Extraction for Hand Movement Recognition Based on Euclidean Distance and Standard Deviation”, Proc. of International Conference on Electrical Engineering/Electronics Computer Telecommunications and Information Technology (ECTI-CON), pp. 856-860, 2010.
- 12.
Christianini, N. and Shawe-Taylor, J., “An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods,” Cambridge University Press, 2000.
10.1017/CBO9780511801389
- 13.
Schölkopf, B. and Smola, A. J., “Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond,” MIT Press, 2002.
- 14.
Phinyomark, A., Phukpattaranont, P., and Limsakul, C., “Feature Reduction and Selection for EMG Signal Classification,” Expert Systems with Applications, Vol. 39, No. 8, pp. 7420-7431, 2012.
10.1016/j.eswa.2012.01.102
- 15.
Marshall, M. M. and Armstrong, T. J., “Observational Assessment of Forceful Exertion and the Perceived Force Demands of Daily Activities,” Journal of Occupational Rehabilitation, Vol. 14, No. 4, pp. 281-294, 2004.
10.1023/B:JOOR.0000047430.22740.57
- 16.
Mon-Williams, M., Tresilian, J. R., Coppard, V. L., and Carson, R. G., “The Effect of Obstacle Position on Reach-to-Grasp Movements,” Experimental Brain Research, Vol. 137, Nos. 3-4, pp. 497-501, 2001.
10.1007/s002210100684
