Continuous Recognition of Multifunctional Finger and Wrist Movements in Amputee Subjects Based on sEMG and Accelerometry

1. INTRODUCTION

Limb amputation is a major cause of disability in the world [1]. The key issues of accomplishing practical upper-limb prosthesis are functionality, controllability and cosmetics [2]. In recent decades, these prostheses controlled by the surface electromyography (sEMG) containing rich information of neuromuscular activity have been applied to replace those controlled by the original motor commands in a noninvasive way [2, 3]. Gradually, these approaches have developed from control of simple functional prosthesis, such as wrist flexion and extension, to multifunctional prostheses [3]. In spite of these technological advancements, the existing prostheses cannot fully meet the actual needs of the amputees in terms of dexterity. Hence, how to control a multifunctional and dexterous prosthesis becomes a tremendous challenge.

Peerdeman et al. [4] indicated that the various approaches proposed by the majority of studies were obtained based on intact subjects. And 4 to 16 electrodes, as the case maybe, would be placed on the mid-portion of the forearm. Among the existing state-of-the-art classifiers, linear discriminant analysis (LDA), artificial neural networks (ANN) and support vector machines (SVM) algorithms play significant roles in predicting the user’s intended movements with high accuracy [5]. In our previous paper [6], we obtained an excellent classification rate at 98.6% on five hand movements using intact subjects. However, it is still an unproved issue that the results acquired on intact subjects can apply equally to trans-radial amputated subjects (TRAS) [7]. Due to the practical and ethical issues lay in this field, the reports on the real-time prosthesis control especially for amputated subjects are rather limited. Moreover, some items should be noted as follows. Firstly, it is a suffering task to think and mimic the finger and wrist movement with amputated upper-limb throughout a few hours. Secondly, different amputation levels impact the performance of classification. Thirdly, amputees differ in learning ability. Thus, different scientific training treatments should be employed on different TRAS. In a general view of literatures using TRAS, Momen [8] showed an approximately 87.5% accuracy rate for 4 arm classes on only one amputee. A recent study included six TRAS and used 64-channel amplifier (12 movements, 87.8% accuracy [9]). Schultz et al. [10] demonstrated an 87.8% accuracy rate on 5 TRAS over 10 movements. It should be emphasized that the results obtained from amputees still remain to be improved.

In recent years, a few researchers paid close attention to accelerometry (ACC), which is relatively low-cost and easily integrated in a prosthetic socket. Zhou et al. [11] indicates that the EMG signals and tri-axial accelerometer mechanomyography signals can reduce the effect of limb position variation on classification performance. Fougner et al. [12] demonstrated that ACC provided useful supplementary for prosthesis controllers.

In this paper, an efficient continuous recognition scheme of multifunctional finger and wrist movements is proposed. Primarily, a root-mean-square (RMS) filter had been employed on the sEMG signals to smooth the data. Considering the practical applications in real-time control, a sliding window was employed. It should be emphasized that majority vote is adopted to eliminate transient jumps and produces smooth output for window-based analysis scheme. Subsequently, we followed the recommendation to use sEMG in conjunction with ACC as control modality [6]. However, the placement and regular orientation of the electrodes would cause redundant information. Principal component analysis (PCA) was hence utilized to reduce the dimension of features and to eliminate the correlation of ACC modality, which improved the classification rate. Six time domains features and one time-frequency domain feature are calculated respectively as the feature vectors of each sEMG segment. The features of sEMG and ACC are then fed into four types of classifiers. Finally, according to the experimental results, the significant features of sEMG and ACC were selected and fused into an integrated feature and then passed into the best-performing classifier, that is, SVM for ultimate classifications.

The rest of the paper is organized as follows: section 2 presents this details of the scheme for data analysis and feature selection is contained in this part as well; the results are demonstrated and assessed with a novel evaluation criterion in section 3 and discussed in section 4.

2. MATERIALS AND METHODS

2.3. Analyzing

2.3.1. Window-based Analysis Scheme

The signals of all channels were split into sliding windows in accordance with the real-time control scheme for multifunction myoelectricity proposed by Englehart et al. [17] as displayed in Fig. (1). For each sliding window, a selected feature vector was calculated and fed to a pattern classifier which could achieve outstanding classification performance. L. Smith et al. [18] indicated that longer window resulted in higher accuracy at the cost of longer processing delays (the value of processing delays in this paper was computed on a 2.5 GHz AMD Sempron^TM X2 processor); meanwhile, the optimal window length was recommended to be in the range of 150-250 ms. Considering the configuration of processor and real-time prosthesis control in future, the overlap increment should be longer than 20ms. Consequently, a 250ms (i.e.100 samples) sliding window with a 25 ms (10 samples) incremental window was considered. Window-based analysis scheme are described in Fig. (1). (For clarity reasons, the figure displays only three out of twelve electrodes. The data is decimated at a sub-sample rate of 10).

Then, the following experiment applied the majority vote (MV) to post-process the successive dense stream of test sequence generated by overlapped segmentation. The majority vote can be described as follow: (1) make a decision for (100 samples) sliding window every 25ms; (2) make N continuous decisions; (3) vote for the N decisions; (4) The decision of the maximum votes represents the final decision. The MV is an effective post-processing mechanism that eliminates transient jumps and produces smooth output for window-based analysis scheme [19]. However, the acceptable control delay in output should be less than 300ms. The parameter k (number of the MV) is constrained within the following inequality:

(1)

Where, increment is the overlap increment between two adjacent windows (25ms). Here, MV with k=3 was adopted to smooth the results (i.e. making a decision every 75ms).

2.3.2. Feature Extraction

Feature selection is an essential stage in sEMG classification. And the capability of presenting the features of different movements and the computational load in real-time control shall be focused. A significant amount of features widely used in myoelectric control design can roughly be divided into three categories, namely time domain, frequency domain, and time-frequency domain [19]. However, it had been proved that frequency domain features usually perform poorly when compared with the time domain features [20].

The most successfully used features in time domain is the Hudgins feature set [20, 21]. Following the recommendations in [20], we had selected six sEMG features including Mean Absolute Value (MAV), Modified Mean Absolute Value 1 (MAV1), Variance (VAR), Root Mean Square (RMS), Waveform Length (WL), and the first four autoregressive (AR) coefficients. Waveform Length is simply the cumulative length of the waveform over the time segment defined as:

(2)

where Δx_k = x_k - x_k-1 (difference in consecutive sample voltage values). MAV1 is a robust modification of Mean Absolute Value with weighting window function (w_n). It is calculated by:

(3)

In addition, according to the classification performance of each single feature, a combination of different single features was also applied to explore more robust features.

Wavelet packet transform (WPT), an extension of discrete wavelet transform, is a more advanced representation in time-frequency domain. The transform is able to decompose a signal using the wavelet function at different levels, resulting in high-dimensional coefficients. In this work, we employed 3-level wavelet decomposition and Daubechies 3 (db3) as mother wavelet, then calculated the variances of each wavelet packet decomposition coefficient of the last level as part of sEMG features [6].

For ACC modality, we applied the mean value (MEAN) that performed excellent in previous work [11]. PCA was applied to reduce the dimension of feature for ACC data and over preliminary trails to ensure the optimal threshold to capture the tradeoff between reduction size and discrimination rate.

2.3.3. Classification

In this section, the classifiers considered here have all been successfully used in conjunction with sEMG signals, including linear discriminant analysis (LDA), k-nearest neighbors (k-NN) with k ϵ {1, 3, 5, 7, 9 and 11}, random forests (RF) and support vector machine (SVM) with a radial basis function (RBF) kernel as following:

(4)

For parameter optimization, the 4-fold cross validation scheme was used to evaluate the best choice of SVM parameters, namely penalty factor C and parameter γ of kernel function. In every experiment, 10-folds cross-validation was conducted. Following the same protocol presented by the authors of the database [13], we also adopted repetition 2 and 5 for testing set, so the remaining four repetitions created training set. Considering computational feasibility, the window sets for training and parameter optimization were subsampled at regular intervals of 3 and 5 respectively (i.e. a window increment of 75 ms and 125 ms). Then, 4 groups were alternately selected as training set to train classifier model and the other 2 groups were used to test the established model. Taking the distributional differences among repetitions into consideration, each of the folds corresponded to one of the four training repetitions instead of random splitting of the folds. In order to find an optimal configuration, parameters were selected using a two-stage grid search via coarse grids, and then fine grids [22]. In the coarse grids, C ϵ {2^-8, 2, 2², 2³, 2⁴, and 2⁵} and γ ϵ {2^-8, 2^-4, 2^-2, 2, 2², and 2³}. In the fine grids, these selected values in coarse grids were examined in a range of ±5%, ±10%, ±15%, and ±20%.

2.4. Evaluation

In addition to classification accuracy, a novel metric was also employed to evaluate the window-based control performance. Movement error rate (MER) proposed in [23] for offline evaluation of myoelectric control systems was utilized in this section. MER is a normalized version of edit distance used to count the minimum number of insertions, deletions and substitutions required for two sequences to match necessarily. Before computing the MER, a key process should be taken to regard movements rather than windows as the atoms by erasing adjacent duplicates in both sequences of true labels and the predict labels. Subsequently, the edit distance (MER) of two processed sequences was measured as defined as followed:

(5)

According to the definition of MER, lower MER implies that the predicted movements (not windows) achieve a higher similarity with true movements, namely the approach obtains better quality of hand movement control.

3. RESULTS

In this paper, all the analyses were conducted under the environment of MATLAB 2014a. A preliminary analysis was implemented to select effective features from the feature list we selected. Fig. (2) shows that the MAV and WPT perform best for single features (75.85±2.58% and 75.12±3.22% respectively), regardless of the training time consumed for WPT. Liner combining the MAV and RMS yields a more robust result than MAV and WPT. Interestingly, the accuracy of the MEAN features extracted from ACC is 86.6±2.59% and thus significantly higher than any of the considered sEMG features. Additionally, Fig. (3) indicates the MEAN feature after PCA obtains a remarkable improvement of approximately 1% (87.65±3.3%) in result than that without PCA. Thus, the first 18 principle components (PCs) have been used in the following experimental validation. The results in Fig. (4). illustrate the average accuracy using either sEMG, or a combination of sEMG and accelerometry, compared with SVM, LDA, Random forests and k-NN classifiers. The SVM with sEMG+ACC modalities performs the best accuracy of 88.72±2.58%, which achieves a significant improvement of almost 13% over sEMG-only and roughly 1.1% over ACC-only. Subsequently, the predicted labels produced by the test sequences of the subject 5 presented in Fig. (5) were compared with the true labels. Finally, MER was calculated from both sEMG and sEMG+ACC as complementary evaluation criterion for window-based analysis (Table 3).

4. DISCUSSION

CONCLUSION

A majority of studies have compared different approaches to develop myoelectric control of upper-limb prosthesis. In this paper, we addressed this problem by classification of 17 functional movements on 5 amputated subjects. Our approach achieves an average accuracy of 88.7±2.6% by combining sEMG and ACC modalities, which confirms that our approach is appropriate for amputated subjects and ACC is an excellent complementary modality for myoelectric control. Furthermore, principle component analysis was employed to reduce the dimension of feature and to eliminate the correlation of ACC signal, which significantly improved the classification performance and reduced MER. In conclusion, the approach we present in this paper boosts the capabilities of dexterous myoelectric hand by hand amputated subjects.

REFERENCES