Get Our e-AlertsSubmit Manuscript
Cyborg and Bionic Systems / 2021 / Article

Review Article | Open Access

Volume 2021 |Article ID 9863761 | https://doi.org/10.34133/2021/9863761

Dongfang Xu, Qining Wang, "Noninvasive Human-Prosthesis Interfaces for Locomotion Intent Recognition: A Review", Cyborg and Bionic Systems, vol. 2021, Article ID 9863761, 14 pages, 2021. https://doi.org/10.34133/2021/9863761

Noninvasive Human-Prosthesis Interfaces for Locomotion Intent Recognition: A Review

Received11 Oct 2020
Accepted22 Mar 2021
Published04 Jun 2021

Abstract

The lower-limb robotic prostheses can provide assistance for amputees’ daily activities by restoring the biomechanical functions of missing limb(s). To set proper control strategies and develop the corresponding controller for robotic prosthesis, a prosthesis user’s intent must be acquired in time, which is still a major challenge and has attracted intensive attentions. This work focuses on the robotic prosthesis user’s locomotion intent recognition based on the noninvasive sensing methods from the recognition task perspective (locomotion mode recognition, gait event detection, and continuous gait phase estimation) and reviews the state-of-the-art intent recognition techniques in a lower-limb prosthesis scope. The current research status, including recognition approach, progress, challenges, and future prospects in the human’s intent recognition, has been reviewed. In particular for the recognition approach, the paper analyzes the recent studies and discusses the role of each element in locomotion intent recognition. This work summarizes the existing research results and problems and contributes a general framework for the intent recognition based on lower-limb prosthesis.

1. Introduction

Lower-limb robotic prostheses, mainly including knee-ankle and ankle-foot prostheses, have achieved a fast development within these years, since it can provide the functional compensation for amputees by mimicking the biomechanical features of joints [15]. By adopting a proper control strategy, the prosthesis can assist an amputee’s daily walking activities, such as walking on level ground or ramps, with low metabolic cost, good gait symmetry, and so on [68].

Researchers have investigated the biomechanics and motor coordination in humans during different locomotion modes, such as walking on different stairs, level ground, and inclined surfaces [9, 10]. The kinematics and dynamics of different joints vary a lot in these locomotion modes, and the prostheses need to mimic the biomechanics of missing joints. Therefore, to set control strategies for prosthesis, it is very important to recognize the human’s locomotion intent. The human’s locomotion intent recognition refers to the interaction among human, prosthesis, and environment. Based on the gait phase (stance or swing phase) and environment (ramps, stairs, etc.) that amputees are in, the recognition, by processing the signals deriving from the human’s residual limb and the mechanical sensors of prosthesis, can instruct the behavior of prosthesis. Some lower-limb’s movements can be viewed as periodical or quasi-periodical in structured environment, and these common periodical activities include level ground walking (LG), stair ascending (SA), stair descending (SD), ramp ascending (RA), and ramp descending (RD). Apart from these periodical movements, there are some nonperiodical movements, such as sitting, standing, stepping over an obstacle, turning around, and walking on uneven terrains or between different terrains. Taking the level ground walking as an example, each gait cycle includes the stance phase and swing phase. The gait phase can be set generally according to the detected gait events (heel strike, heel off, push off, toe off, etc.). Currently, the most used finite-state machine control method for robotic prostheses is based on these states (detected gait phase and events) [6, 7], as the set function of prosthesis in each state is different. For example, during the swing phase, the prosthesis needs to achieve foot clearance and reset to a desired equilibrium position [6, 7]. During the stance phase, the prosthesis needs to propel the body upward and forward [6, 7]. In addition to the finite-state machine control method, some alternatives are developed based on the estimation of the continuous gait phase that increases monotonically in each gait cycle.

The human’s intent recognition, including locomotion mode recognition, gait event detection, and continuous gait phase estimation, is the necessary prerequisite to set a control strategy. Thus, it has attracted a multitude of groups to conduct related studies and got some good results [1118]. Several research groups have given reviews about the human’s intent recognition for a control perspective [19, 20]. Tschiedel et al. have reviewed the sense for enhancing lower-limb prosthesis control and introduced the different types of sensors in locomotion recognition [21]. Novak and Riener have conducted the survey of sensor fusion methods for the human’s intent recognition in wearable robotics [22]. Windrich et al. have also presented a review on design issues and solutions found in active lower-limb prostheses [23]. This paper presents a review of the human’s intent recognition based on lower-limb robotic prostheses from the recognition task perspective: locomotion mode recognition, gait event detection, and continuous gait phase estimation.

The paper is aimed primarily at the approach and research progress, challenges, and future prospects in locomotion intent recognition of lower-limb robotic prosthesis. The approach includes the raw signals’ preprocess, classifier training, and validation method and recognition test performance. Based on this review of the recognition approach, the current research progress and results are listed by summarizing the studies. The challenges of the approach and research are also discussed in the paper.

2. Recognition Tasks

This work focus on the prosthesis users’ intent recognition research. Most studies concerning intent recognition are conducted in structured environment, such as stairs, ramps, and level ground, as shown in Figure 1. The main intent recognition tasks of lower-limb prosthesis can be summarily divided into locomotion mode recognition, gait event detection, and continuous gait phase estimation, as shown in Figures 2(a)–2(c). The daily activities of lower-limb prosthesis include but not limited to these modes: St, LG, SA, SD, RA, and RD. These locomotion modes (each gait cycle of prostheses does not include more than one mode) can be defined as steady modes. Sometimes, one gait cycle may include more than one mode. Namely, there is locomotion mode transition during this stride period. Nowadays, more and more researchers focus their studies on the continuous locomotion mode recognition (i.e., combining steady modes with transitions between different steady modes) [12, 15, 25, 26]. These transitions are corresponding to the two-way arrows in Figure 2(a). For the given six locomotion modes, there mainly exist ten transitions in daily activities: from LG to St/RA/RD/SA/SD and from St/RA/RD/SA/SD to LG, as seen in Figure 2(a). For the healthy people, they can transfer to one new locomotion mode naturally. But for the prosthesis users, when they transfer their prosthesis leg to a new mode, the prosthesis needs to know the transitions in advance in order to make the corresponding responses [16]. Besides, the lower-limb recognition tasks also include stepping over an obstacle [12, 27] and turning around [28]. The gait event plays an important role in prosthetic finite state machine control [7, 29, 30], which relies on the detected gait events 19 to trigger the transitions of control strategies. Figure 2(b) describes some gait events in each gait cycle, and one gait cycle starts from the heel strike to heel off and toe off and then ends at the next heel strike. The gait phases are generally divided into the stance phase (from heel strike to toe off) and swing phase (from toe off to next heel strike). In addition, more sub phases can be set and divided according to the need to realize more elaborate control [7, 31, 32]. Finite state machine has limitations in the smoothness and robustness of control [33]. Another alternative control method is based on the estimation of the continuous gait phase, usually defined as a number that increases monotonically from 0 to  rad in each gait cycle, as shown in Figure 2(c).

3. Locomotion Mode Recognition Method

For the locomotion mode recognition, one important step is to build classifiers based on the sensing signal data to realize modes’ classification. The process of building classifiers is customarily called as classifier training. Based on the trained classifiers, the recognition test is conducted next. The recognition performance is the critical target, and the performance metrics of the recognition test are generally the accuracy and the time performances (decision time [15] (the time required by the classifier to reach a decision [19]) and delay between gait transitions). The different types of classifiers [29, 34, 35], signal sources [13, 15, 3638], algorithms [13, 39, 40], optimization methods (sliding window), feature extraction and feature selection [16, 26, 41], training and validation, and recognition test will be discussed in this work.

3.1. Heuristic Rule-Based Classification

The types of classifiers can be divided into two types: heuristic rule-based classification based on a set of rules and automated pattern recognition based on machine learning and statistics [19]. Heuristic rule-based classification is an effective method in locomotion mode recognition and can be easily understandable [29, 34]. For most conditions, the establishment of rules is to find the boundary or division surface between two locomotion modes or gait events. It is not hard to build the rules based on some collected sensing data in advance. The criteria to make a set of rules are unfixed, and they can be made according the tasks’ features and the experimenter’s experience by analyzing the mathematical features of signals. Trial and error or feedback can also provide instruction to make and improve rules.

For locomotion mode recognition, some studies have achieved quite good performances based on this method. Yuan et al. adopted the fuzzy logic-based threshold rules to identify the terrains for transtibial amputees [29]. Li and Hsiao-Wecksler also adopted the rule-based method to recognize the level walking, stair ascent/descent, and ramp ascent/descent by evaluating the slope of the flat surface and elevation of the foot in one gait cycle [34].

Though the rule-based classification method is simple and effective, sometimes the establishment of rules may be complicated as the gait modes increase. More gait modes will bring more parameters to tune for the establishment of rules, so performance will deteriorate. Besides, the adaptations of rules are also a notable question over time, as well as users. The accuracy may be assured since there are relatively distinct signal features between different modes; however, the noise and smoothness of signals will cause some errors. The decision time is sometimes very small, since there are just several judgement rules. But for terrain or locomotion mode recognition, there is at most a one-stride delay based on the set rules, since the rules work at several discrete points of the gait cycle [29, 34].

3.2. Pattern Recognition

Pattern recognition is widely used in lower-limb locomotion mode recognition, which is rooted in the fields of machine learning and statistics [19]. Pattern recognition not only is used to identify several fixed gait modes by building map relationships [12, 15, 16] but also can be used to detect the continuous gait phase by regression fitting [42, 43]. In this section, we focus mainly on the locomotion mode recognition based on pattern recognition. Supervised learning is adopted for locomotion mode recognition, which includes classifier training and validation and recognition test, as shown in Figures 3(a) and 3(b). The collected data set (training data with corresponding labels) are input to the classification algorithm to build classifiers, as shown in Figure 3(a). The classifiers will output recognition result when signals are input to the classifiers, as shown in Figure 3(b). Tucker et al. have depicted the benefits and the shortcomings of pattern recognition as follows. The clear benefit of using an automated classifier over one based on heuristic rules is that data from a multitude of sensors can be input to the classifier, from which additional features may be computed and used to make classification decisions that are less biased and potentially more accurate due to the high-dimensional input [19]. The biggest shortcoming of this approach is the necessity of properly classified training data for all of the desired activities and the transitions between them, preferably incorporating sufficient variability such that the classifier will perform well in real-world scenarios [19]. Furthermore, optimal classifier performance often requires training data from the user himself, which may be somewhere difficult, impractical, and impossible to obtain [39, 44].

For locomotion mode recognition in a robotic prosthesis scope, what needs to be noted first is the input sensing signals. Different sensing signals can record different functional and physical information of the lower limb’s movement. For rule-based classification, raw signals may show distinct differences between gait modes and can be used for classification. However, for pattern recognition, the raw signals need to be preprocessed by extracting time- or frequency-domain features in sliding windows [12, 15, 35]. In addition to the raw signal preprocessing, the other related problems about pattern recognition can be summarized from training, validation, and recognition test. The types of input signals, sliding window, feature extraction and selection, label method, and classification algorithms can all affect recognition performances. Besides, the corresponding subproblems are also summarized and reviewed, including training techniques, online or offline recognition, the optimization method of recognition accuracy, and time performance.

3.2.1. Sensing Signals

The sensing signals in the lower-limb prosthesis scope can be divided into two kinds: invasive sensing signal and noninvasive sensing signal. The invasive sensing is not the scope of this work. The noninvasive sensing signals derive from the prosthesis itself and the muscle of the residual limb. As is known, muscles behave as the actuators of the sensory motor system and they contain abundant motion information [19, 45, 46]. Muscle signals could directly and deeply reflect the human’s locomotion intent. Surface electromyography (sEMG) sensors can record the electrical potential of muscle which is generated by muscle cells and is widely used in locomotion mode recognition in lower-limb prosthesis research [12, 4751]. The electrodes of sEMG sensors provide a noninvasive technique for measurement and detection of electromyography signal. The theory behind these electrodes is that they form a chemical equilibrium between the detecting surface and the skin of the body through electrolytic conduction, so that current can flow into the electrode [52]. Therefore, in practical measurement, the electrodes must be attached to the limb’s skin tightly. The sEMG sensing method has some problems and limitations. The measured sEMG signal is weak and nonstationary [53]; besides, it is easy to be contaminated by motion artifacts and muscle fatigue, shift of the electrodes, and crosstalk between nearby muscles [54, 55]. A new noncontact capacitive sensing method has been proposed to measure the relaxation and contraction of muscle [35]. The human’s limb and the metal electrode could be viewed as the two electrodes of one capacitor. The dielectric layer (for example, the silica gel layer) is placed between the metal electrode and human limb, which consists of one equivalent capacitor. During limbs’ locomotion, the contraction and relaxation will cause the change of the relative area and the distance between two electrodes. By measuring the cycle time of charging and discharging, the capacitive signals are recorded [35]. The capacitive sensing method has its specific advantages, since it does not need to be attached to the skin and can void some contacted affection relative to sEMG. Besides, it has good and obvious signal repeatability and stability [42]. The capacitive sensing method has been applied to recognize the locomotion mode and proven feasible [13, 36]. In addition to the muscles’ signals of the residual limb, the robotic prostheses are integrated with different kinds of mechanical sensors: goniometers, accelerometers, gyroscopes, magnetometers, inertial measurement units (IMUs), load cell, strain gauge, and so on. All these mechanical signals can be used for locomotion mode recognition or gait phase detection [14, 15, 5660]. Mechanical sensors are easy to be integrated with prosthesis than sEMG. However, compared with sEMG, what the mechanical sensor measures is the already happened movement information, so there exists delay compared with sEMG [61].

In addition to the mentioned sensing method based on the residual limb and prosthesis, electroencephalography (EEG) signals of the brain can illustrate the locomotion information. The EEG method can monitor electrical activity across the brain with high information content [62]. Now, there are quite a few studies in lower-limb prosthesis based on the EEG method [63]. The EEG-based approach has shown some effects in locomotion intent recognition of lower-limb prosthesis [64]. Compared with sEMG, the EEG signal is a nonstationary signal and has low resolution in lower-limb locomotion intent recognition. Besides the mentioned sensing signal types, others sensing signal can also be used for locomotion mode recognition. Tschiedel et al. have summarized the different sensors in lower-limb prosthesis locomotion recognition and control [21], including distance and depth sensors, kinematic sensors, and others. Different types of sensing signals can provide different physical information corresponding to the same locomotion mode. In wearable robotics for locomotion recognition, sensor fusion methods have been proven to be an effective way to improve recognition performance [22, 65], such as the fusion of capacitive sensors and IMUs [13] and the fusion of sEMG and mechanical sensors [12, 16, 66]. In this work, the sensor fusion denotes multitype sensor fusion not the multisensor of one type.

3.2.2. Preprocessing of Sensing Signals

Raw sensing signals need to be preprocessed to remove the noise and provide more effective information before it is used for locomotion mode recognition. For different signals, especially for sEMG and capacitive sensing signals, which are easily contaminated by artifact and noise [13, 35, 52, 6769], they need signal filter operation first. For some other signals, especially mechanical sensors, whose signals are quite robust than those of sEMG, their filter process is operated easily. Therefore, this part pays more attention to the filter of the limb’s signals (i.e., sEMG and capacitive sensing signals). sEMG signals are weak and need to be amplified first [52, 53]. Noise plays a major role in hampering the recording of the EMG signal. For this purpose, the signal has to be properly filtered, even after differential amplification [70]. The study [52] has depicted the different filter designs based on the noise frequencies. Low-frequency noise derives from amplifier DC offsets, sensor drift on the skin, and temperature change, and it can be removed using a high-pass filter. High-frequency noise is caused by nerve conduction and high-frequency interference from radio broadcasts, computers, cellular phones, etc., and it can be deleted using a low-pass filter [52]. Some groups have developed the prosthesis studies based on filtered sEMG [12, 47, 71]. Huang et al. have adopted the filter from 20 to 420 Hz in the locomotion mode recognition of the prosthetic leg [12]. The design of low- and high-pass filters can also refer to the study of the European SENIAM project, which suggests that the lower cutoff frequency of the filter is generally 10 or 20 Hz and the upper cutoff is between 400 and 500 Hz [69]. Capacitive sensing signals do not need to be amplified for its relative strong and robust signals compared with sEMG signals [35, 42]. The noises of raw capacitive sensing signals are comprised of low-frequency drifts (lower than 0.1 Hz), random impulses, and high-frequency noise [35]. Accordingly, the studies [13, 35] have used three filters in series (a median filter, a first-order DC-notch filter, and a second-order 10 Hz low-pass Butterworth filter) to regulate the capacitive sensing signals by removing the baseline shifting and the high-frequency noises.

The filtered sEMG signals, capacitive sensing signals, and mechanical signals (more studies tend to fuse the two or all of them) are sampled at individual frequency and then packed together to form data streams. For each frame of signals, it contains quite small information, making it hard to reflect the signals’ dynamic features. Therefore, a sliding window is adopted to extract the signal features, as shown in Figure 4. The data in the window is refreshed continuously as data sampling goes, which also can be viewed as the window is sliding. The length of the sliding window is the number of its contained data frames. Here comes one question: how to decide the length of the sliding window. In locomotion mode recognition of robotic prosthesis, several studies have validated the relationship between the length of the sliding window and recognition accuracy [35, 37, 72]. Besides the length of the sliding window, the sliding increment from one window to the next window is also an interesting index [37]. If the sliding window is smaller or bigger than the length of the window, it means that there is an overlap or no-overlap area of two adjacent windows, as shown in Figures 4(a) and 4(b). Now, more studies choose the small sliding increment to make the current window overlapped with the previous window [12, 26, 73], which allows more frequent commands to the robot and fewer sudden changes in sensor fusion output [22]. Xu et al. have adopted a sliding window in locomotion mode recognition of transtibial prosthesis whose length is 250 ms and increment is 10 ms [15]. Huang et al. have developed the locomotion mode recognition for prosthetic legs, and the length and increment of the sliding windows are 150 and 12 ms, respectively [12].

Most studies about locomotion mode recognition have attempted to extract features from raw signals rather than using raw signals directly to the recognition studies. The aim of conducting feature extraction is to acquire more useful information to distinguish one locomotion mode from another. Elhoushi et al. have reviewed the feature types in their survey [74], which are listed as follows: (1) statistical features, (2) time-domain features, (3) energy, power, and magnitude features, (4) frequency-domain features, and (5) other features. As the types of input signals vary, therefore the features of different input signals are different [12, 15, 35, 75]. For sEMG signals, there are some well-established feature types for EMG signals: autoregression coefficients, time domain, frequency domain, or time-frequency domain [12, 7577]. Tkach et al. have adopted some time-domain features in their study, and the main features of the time domain are mean absolute value, zero crossings, slope sign changes, waveform length, Willison amplitude, variance, -order, log-detector, EMG histogram, autoregression coefficient, and cepstrum coefficients (the detailed means of each features can be gotten in the study) [78]. Huang et al. have used the different feature types of sEMG (the mean absolute value, number of slope sign changes, waveform length, and number of zero crossings) and mechanical signals (the maximum value, minimum value, and mean value) in their study [12, 79]. For capacitive sensing signals, as it is a new sensing method, the selection of statistical or time-domain feature types is introduced and tried. Chen et al. have selected eight features (the mean, root mean square, standard deviation, maximum, minimum, interquartile range, mean absolute deviation, and first derivatives) in their locomotion recognition research based on the capacitive sensing method [36]. Zheng et al. have developed statistical and time-domain features for capacitive signals: mean value, maximum, minimum, standard deviation, several differential features, and correlation coefficient. For mechanical signals, their statistical features and time-domain features are commonly used in recognition. Xu et al. have extracted the five features (the mean value, maximum, minimum, standard deviation, and the differential signals) of two IMU signals in their recognition of continuous locomotion modes for robotic transtibial prosthesis users [15]. For signal fusion (e.g., capacitive signals and mechanical signals), Zheng et al. have calculated six features (the mean, standard deviation, maximum, minimum, sum of signals’ absolute value, and differential value of signals) for capacitive signals and four features (mean, standard deviation, maximum, and minimum) for mechanical signals of prosthesis [13]. The feature types are not fixed, and there are no clear criteria for that. Therefore, the researchers can try different types of features (trial and error), only if they are helpful for specific research tasks. As to locomotion recognition, the accuracy should be one critical target to consider when some types of feature are chosen to be candidates.

For the supervised pattern recognition, one importance part is to label the training data to their corresponding modes to form a training data set [80]. In fact, label is critical but not difficult in locomotion recognition research. Labeling data must be unmistakable for training classifiers. The labeling method can be manual or with assistance according to the experimental tasks [35, 39]. Classification algorithms play an important role in locomotion mode recognition, and a plenty of algorithms have been unitized, including Dynamic Bayesian Networks (DBN) [81, 82], linear discriminant analysis (LDA) [35, 37, 41], Quadratic Discriminant Analysis (QDA) [41, 83], Gaussian Mixture Models (GMM) [35, 39], Support Vector Machines (SVM) [12, 15], Artificial Neural Networks (ANN) [37, 40], and Convolutional Neural Network (CNN) [56]. The advantages and disadvantages of these classifiers and the mechanics of the classification algorithm can refer to some works about machine learning and statistics. Some studies have conducted the recognition accuracy comparisons of different classification algorithms [13, 15, 41]. As the complexities and classification principles of algorithms vary greatly, the selection of the algorithm will bring improvements for some specific performances at the cost of deteriorating other performances. For example, when choosing the complicated classification algorithm to improve recognition accuracy, it may bring more time cost to train classifiers. With the benefit of integrated circuits, a hardware device can process the data with a complicated algorithm with high speed, big storage, and low power consumption performances [84, 85].

The sliding window’s length and increment and the feature types can also be optimized to improve locomotion recognition accuracy [37], as they have been mentioned above. As is known, signal feature extraction will form a feature vector with big dimension. We take the one study as an example. In the study [13], capacitive sensors (six channels’ capacitive signals) and mechanical (ten channels’ mechanical signals) sensors are adopted. After raw data are preprocessed, they form one feature vector with 76 elements. In the recognition, if more sensors (sEMG, capacitive sensors, accelerometers, gyroscopes, and so on) are fused, they will form feature vectors with bigger dimensions. In addition to the dimension of the feature vector, each value in one feature vector contributes to recognition differently. The invalid feature values may contribute little and even harm the recognition. Therefore, feature selection is essential and important for improving recognition performance and reducing computation time since it can reduce the feature vector’s dimension by removing some invalid feature values. The selection of the classifier is generally considered to be less important than the selection of features [86, 87], where the effect of the classifier type on accuracy is generally small in some prosthetic hand studies [88, 89] and lower-limb movement [41]. Feature selection is aimed at choosing the more suitable subset of features which shall result in the better recognition accuracy [90]. One widely used method for feature selection is sequential feature selection (SFS) [91, 92]. SFS can be divided into sequential forward feature selection (SFFS) and sequential backward feature selection (SBFS). SBFS is an opposite operation to SFFS. SFFS, aimed at improving the recognition accuracy continuously, starts with an empty feature set and iteratively retrains the classifier by adding a new feature which can increase the recognition accuracy best to the feature set. When the accuracy can get obvious improvement that meets requirement or starts to increase very slowly or even decline, SFFS stops retraining and the selected subset of the feature vector is the new feature vector with smaller dimensions than the previous feature vector. SFFS has been applied in the lower limb’s locomotion mode recognition on healthy people [41] and amputees who wear prosthesis [13, 15]. Feature transformation is also a dimension reduction method and can be viewed as another feature selection method, which can create a new feature vector by a mathematical transformation [93] and is helpful to improve recognition performance. Feature transformation methods consist of linear discriminant analysis (LDA) [94, 95] and principal component analysis (PCA) [94, 96, 97], factor analysis, and nonnegative matrix factorization [74]. LDA and PCA are two commonly used methods in the locomotion mode recognition [16].

3.2.3. Training and Validation

Locomotion mode recognition is aimed at the final recognition test, which is on the basis of training and validation. Different from the signal preprocessing, training techniques are sometimes dependent on experimental tasks. One main technique of training is how to build the training data set, especially for some data representing the transition process from one locomotion mode to another locomotion mode [26]. For example, from level ground to ramp or stair, there are several transitions among locomotion modes. In building the training data set, the data belonging to the transitions must be treated carefully, since they can result in big influence on recognition performance [26]. It must be taken into consideration whether to choose these data or remove these data. If the data corresponding to transitions are treated as two locomotion modes, the concern is how to decide the two modes’ labels of these data. These studies have adopted reasonable divisions to the training data set and get quite good performances in the test of a steady mode (level ground, ramps, and stairs) and transitions (level ground to ramps or stairs, etc.) [12, 13]. Young et al. have conducted the training method research in more detail and concluded that the training data set containing data corresponding to transition will result in better recognition performance [26]. In addition to that, Young et al. have also concluded that recognition at several critical moments of each gait cycle would get accuracy improvement than continuous recognition at each moment of the gait cycle.

A phase-dependent classifier is an effective design in improving lower-limb locomotion mode recognition performance [12, 13, 15]. As is known, one gait cycle can be divided into the stance phase and swing phase (as shown in Figure 1(b)) or more phases: early stance, middle stance, late stance, swing phase 1 (knee flexion), and swing phase 2 (knee extension) [19]. As some movements of the lower-limb are periodic or quasi-periodic, the sensing signals vary periodically or quasi-periodically. Huang et al. have studied the EMG signals; although time-varying but quasi-cyclic, the muscle activation patterns for the same locomotion mode are similar at the same gait phase [37]. They have assumed that the pattern of EMG signals had small variation in a short time window. Hereby, they have designed phase-dependent EMG pattern classifiers to recognize prosthesis users’ locomotion modes [37]. The phase-dependent method provides researchers a lot of benefits to design different classifiers corresponding to different phases. Different phases have their specific classifiers with better performance than fixed classifiers in the whole gait cycle, which contributes to the lower-limb locomotion mode recognition [12, 13, 15, 73].

Based on the training data set and phase-dependent method, the classifier must be trained and validated by setting aside an independent validation set from the initial training data set for the last purpose (generalization test). Cross-validation can be an effective way to estimate generalization error. Choosing what fraction of the data should be used for training and for validation is an open problem. For the relatively small training data set, leave-one-out cross-validation (LOOCV) has been used in each phase for more precise estimation of the classification error [98]. Huang et al. have adopted the LOOCV in their studies for locomotion mode recognition [37]. Zheng et al. have conducted the fraction research of training and validation for locomotion mode recognition and got the conclusion that using the leave-one-out cross-validation procedure can achieve the best performance.

3.2.4. Adaptation of the Classifier

The adaptation of the classifier means whether the classifier can maintain its recognition performance as conditions change, such as each donning and doffing of the prosthesis, long time wearing, and new participants joining. All these problems are related to the classifier’s adaptation. The simplest and most used recognition approach is utilizing the nonadaptive classifier with time-invariant property for locomotion mode recognition. The time-invariant classifier can achieve good performance with limited conditions: subject-dependent, short-interval, and fixed experimental protocol. For most recognition studies of lower-limb locomotion mode, the subject-dependent classifier can achieve good performance for specific subject and it cannot fit another one well generally, because of the subject’s individual difference in physical functions, body conditions, and so on. As the zero and temperature drift exist, sensing signals will show some time-variant property even with periodical rectification. In addition, for the sEMG sensor, its signals are easily affected by muscle’s morphology: electrode shift, sweat, muscle fatigue, etc. [54, 55]. Furthermore, when the experimental protocol changes, such as speed, inclination degree of ramp, and height of stairs, sensing signals also show different signal features. All these factors can cause the decline of the classifier’s performance. One solution to these problems is to design one adaptive classifier.

Adaptive classifiers are first proposed in the upper-limb field based on sEMG signals in the studies [99101]. Recently, researchers have adopted the adaptive design to the lower-limb prosthesis scope to adapt to sEMG pattern variations over time, caused by physical and physiological changes [73, 102]. Du et al. have developed an adaptive locomotion mode recognition framework in dealing with gradual sEMG magnitude change [102]. The kernel of their adaptive algorithm is to add test data into the training data set to retrain the classifier as time goes, and the retrained classifier (i.e., updated classifier) is then applied to test the new data [102]. Therefore, the correct label of the added data into the training data set is very critical since they will be used to retrain the classifier. Spanias et al. have proposed a strategy for labeling data by identifying the mode of the users’ most recent stride and then providing a label for the corresponding data [103]. They have conducted a preliminary study with an adaptive recognition system for novel users using a powered lower-limb prosthesis and achieved good effects: compared to a nonadaptive system, the adaptive system can reduce the number of errors by 32.9% [104]. Researchers have also developed the user-independent research by pooling data from a large subject group and got high accuracies of gait mode identification for a novel subject [24]. Spanias et al. have compared the classification types (user-independent, partially dependent, and user-dependent) and get some results. Subsequently, Spanias et al. have developed the across-user adaptation for a powered lower-limb prosthesis [105]. Up to now, Spanias et al. have combined the adaptive intent recognition algorithm with prosthesis control and enabled incorporation of neural information over long periods of use, allowing assistive robotic devices to accurately respond to the users’ intent with low error rates by updating (retraining) classifiers. The research about the adaptive classifier for lower-limb prosthesis is still at an early stage, which needs researchers’ continuous efforts.

3.2.5. Recognition Test

The recognition test is the new preprocessed test data as input flows to the classifier and then it outputs the recognition result, as shown in Figure 2(b). For most lower-limb locomotion mode recognition, researchers have developed the offline and real-time recognition tests based on offline trained classifiers. Offline analysis can reflect the performances of the classifier, but the real-time test has more practical values, which need more attention to be paid. The decision time of the real-time recognition process must be shorter than the sliding window’s increment, so as to leave enough time for the following control response and avoid the time collision with the next recognition decision [72], which makes demand of the processing capacity of computing systems. Currently, Zhang et al. have developed the design and implementation of the neural-machine interface for artificial legs to identify users’ intent in real time very early [106], and they have also conducted the real-time recognition in MATLAB [73]. Real-time recognition on board may be more valuable and practical to the control of prosthesis than it is in MATLAB. Some latest studies have developed the on-board recognition study to recognize the user’s intents [15, 16]. Of course, no matter how the recognition test is performed (online or offline), the tested performance is decided by the training and validation fundamentally. No matter what the adopted recognition algorithm is, there may always exist some recognition error. Still, we can adopt some approaches to conduct postprocessing of recognition result. Majority voting (MV) is a widely used postprocessing approach in locomotion mode recognition [12, 72, 107]. The MV approach utilizes recognition results of multiple adjacent analysis windows to produce more accurate recognition decisions. Huang et al. have utilized the MV approach and proposed an enhanced MV by increasing the number of voting decisions each time rather than using the fixed-point majority voting in continuous locomotion mode recognition for prosthetic legs based on neuromuscular-mechanical fusion [12]. However, one study points out that MV does not have a practical effect on real-time task performance [108]. Chen et al. have adopted some ideas to improve the MV approach by adding weight value for each decision, which is determined by the posterior probability of the recognized decision with the LDA classifier, and have gotten good performance in their real-time locomotion recognition [109]. Simon et al. have proposed a new postprocessing approach (i.e., decision-based velocity ramp) to reduce error of recognition in the research of the prosthesis hand, which is related to speed with which the speed wearable robot moves [108]. Their research has shown significant performance advantages over the majority vote, improving task completion speed in amputees with powered prostheses.

For some steady modes, such as level-ground walking and ramp ascending, the main metric to evaluate the recognition performance is the recognition accuracy. For the users, they often face the transitions between different locomotion modes. The recognition of continuous locomotion mode is more meaningful since it can provide the transition information to instruct the robotic prosthesis to adjust the corresponding control automatically. To evaluate the recognition performance of continuous locomotion, the accuracy in the steady mode and the delay or accuracy in the transition period are adopted [12, 15]. Hargrove et al. have acquired low error rates for transitions between different locomotion modes based on adaptive classifiers. Xu et al. have conducted one real-time test for eight transitions and got both advanced and behindhand recognition before the defined critical gait events in transition periods [15]. Huang et al. have conducted an offline test and analyzed the five transitions, and their research has achieved at least advanced 150 ms before the defined critical gait events [12].

4. Gait Event Detection

The finite-state machine method for prosthesis control needs to know the current state (i.e., the gait phase or gait event) that the prosthesis is in [6, 7, 30]. The detection methods for the gait phase or gait event can be classified into two types: pattern recognition and threshold decision, and the characteristics of each method can be seen in Table 1.


MethodPattern recognitionThreshold decision

ComplexityComplexSimple
OperabilityModerate-hardEasy
Detection accuracyHighHigh
Detection timeMoreLess
Sensor replaceYesNo

The pattern recognition method can realize gait phase or event detection using alternative sensing signals. For example, recognizing the swing and stance phases and using the pattern recognition method based on inertial or capacitive sensing signals can replace the load cell [110, 111], which is one big advantage, since a load cell sensor is susceptible to mechanical fatigue and has limited working lifespan. Apart from this, pattern recognition is a relatively complicated method and cannot work better than sensor signal-based threshold rules for detecting some gait phases.

The threshold decision method is based on the output sensor signals directly, and it is an effective, understandable, and easy-to-operate method to realize gait phase or event detection. This method can directly decide the gait event and then divide one gait cycle into several different gait phases using the signals of different sensors (i.e., load cell [5, 7], strain gauge [56], and ankle of knee angle sensors [6, 30]). Mendez et al. have adopted joint angles, ground reaction force, and time thresholds to divide each gait cycle into four gait phases to make a finite-state machine controller [27]. The threshold method has good time efficiency since its low computation complication, and one study have given that the threshold method takes less than 1 μs to recognize the swing and stance phases by the prosthetic load cell’ signals [15].

To recognize the gait phase, some gait events (for example, the start and end of each gait phase) play an important role in prosthesis control. These critical gait events are corresponding to the start of a new control strategy and also the transition timing between two different control strategies. The difficulty in gait event detection, at present, is how to realize its adaptive detection, such as at different walking speeds, on different terrains, and for different prosthesis users. One latest study has tried to realize the adaptive detection for maximum dorsiflexion timing of prosthesis during walking at different speeds and on different ramps for three prosthesis users by updating the detection model and threshold rules using the several previous gait data [112]. This gives an indication to realize detection adaptation, which requires more attempts and further explorations.

5. Continuous Gait Phase Estimation

Different from dividing one gait cycle into several gait phases, a continuous gait phase increases monotonically from 0 to  rad (or 0 to 100%) in each gait cycle or in one specific gait period (for example, in the stance phase period) when the prosthesis is in periodical or quasi-periodical movement.

For the moment, there are three popular methods explored in the continuous gait phase estimation of lower-limb prosthesis. The first method is to calculate the average duration of several previous gait cycles as the denominator and then calculate the time percent (i.e., gait phase, from 0 to 100%) relative to the average duration in each gait cycle. The second method is designing or utilizing a specific algorithm to estimate the continuous gait phase, such as adaptive oscillator [113] and extended Kalman filter [17]. Xu et al. have used the IMU signals to estimate the continuous gait phase of transtibial prosthesis, and their study results have shown some estimation adaptation to different walking conditions when using the adaptive oscillator method [113]. Thatte et al. have used the extended Kalman filter to estimate the continuous gait phase in the stance period (namely, starting at heel strike (0%) and reaching 100% (precisely at toe off)) based on IMU and angle signals of powered transfemoral prosthesis [17]. The third method is based on the polar angle method [114]. Holgate et al. have found that the polar angle between the tibia angle and its scaled angular velocity has an invertible relationship with the gait phase and is not subject-dependent, and they have built a fitted function between the polar angle and gait phase and realized the continuous gait phase estimation [114]. Quintero et al. have computed the continuous phase in their study utilizing thigh angular position and its corresponding integral to form a well-defined thigh orbit [18].

Although all the three methods can realize continuous gait estimations, each method has its advantage and disadvantages. The first method is easy and simple; however, the estimation accuracy may encounter decline since there may exist difference between the average duration and the current gait cycle time length. The second method, for example, adaptive oscillator, can get high estimation accuracy and good adaptation to steady locomotion mode (different speeds and terrains [113]). However, its performance may be affected by variations in gait and encounter decline when applied to unsteady locomotion. The extended Kalman filter is able to quickly adapt to step-to-step gait variations [17], but its estimation accuracy still needs improvement. The third method has good adaptations due to its subject-independent features, whereas it is sensitive to step-to-step gait variations and its performance is susceptible to signal drift and integral drift. Though these methods have limitations, the prosthesis control based on continuous gait has shown its robustness and smoothness, which is one good alternative to improve prosthesis control.

6. Discussion

This paper has reviewed the state-of-the-art research progress about lower-limb intent recognition from the locomotion task perspective. Human intent recognition is important to make lower-limb prosthesis users finish their daily activities more naturally and comfortably. The human’s intent recognition, including (not limited to) locomotion mode recognition, gait event and phase detection, and continuous gait phase estimation, is conducted based on sensing signals. sEMG, as one widely used sensing method, has a contact problem between the electrode and skin, which is vulnerable to sweat and easy to cause skin indentation and even fester. Jeong et al. have developed one capacitive epidermal electronics for electrically safe, long-term electrophysiological measurements, which can improve the measurement of sEMG [115]. This capacitive epidermal electronics are thin, soft, and stretchable, which enables conformal contact on the surface of the limb, with the ability to accommodate skin deformation in real time [115]. This capacitive epidermal electronics has provided a new approach in sensor design, while its practical effect in lower-limb prosthesis has not been validated now and is worth exploring. The capacitive sensing method can detect muscles’ relaxation and contraction, and it has shown some effects in improving recognition. Currently, the capacitive sensing method is still in the preliminary stage in decoding the human’s movements. In addition to improving the sensors’ designs, multisensor fusion has also shown its advantages. More sensors can provide more locomotion signals; therefore, it is helpful to improve the human’s intent recognition. However, some sensors are hard to be integrated with prosthesis for their big sizes (for example, depth camera); besides, more sensors will make prosthesis weight increment. Both of these will make prosthesis redundant, decrease the assistance performance, and may increase difficulties in wearing. All the mentioned above can be summarized as an approach that roots in sensors to improve the human-prosthesis interface and enhance the human’s intent recognition. Besides, one another approach is to try to change or improve the human. One approach that has been validated in intent recognition is targeted muscle reinnervation (TMR) [81]. Targeted muscle reinnervation (TMR) is a new approach to rebuild the lost electromyographic signals for amputees and is helpful to improve locomotion mode recognition performance and profit prosthesis control with sEMG decoding [81]. This approach is one good exploration, and it provides a new way of thinking to improve intent recognition.

The current locomotion intent recognition performances cannot totally meet the demand in prosthesis control, since there always exist recognition errors. The effects of recognition errors to prosthesis control also need further analysis. One group has studied the effects of recognition errors on volitional control of powered above-knee prostheses and concluded that some errors cannot affect the control of prosthesis and some errors can be avoided by adopting some control strategies based on recognition [32]. Nevertheless, there are some recognition errors that can deteriorate the prosthesis performance and need to be dealt with. Future studies need to adopt a better method and improve recognition techniques to enhance intent recognition performance.

Most of the current studies about locomotion intent recognition are conducted in the structured environment, which has a distance to practical application in the real world. In addition, the adaptation problems of locomotion intent recognition have not been well solved. The adaptation problems are relatively complicated, which proposes requirements from hardware to software (sensors, recognition algorithms, control strategies, prostheses design, etc.), which will call for more interdisciplinary cooperation and research collaborations.

7. Conclusion

To set control strategies for lower-limb prosthesis users’ daily activities, it is important to recognize the user’s locomotion intent. For the lower limb, gait event detection, continuous gait phase estimation, and locomotion mode recognition are the main recognition tasks. The paper develops the recognition review specifically for the human’s locomotion intent recognition based on lower-limb prosthesis, reviews the state-of-the-art human’s intent recognition techniques, and summarizes and analyzes the general framework for the intent recognition with lower-limb prosthesis. This work also summarizes the research progress and points out the challenges and future prospects in the human’s locomotion intent recognition based on lower-limb robotic prosthesis.

Conflicts of Interest

The authors declare no conflict of interest.

Acknowledgments

This work was supported by the National Key R&D Program of China (Nos. 2018YFB1307302 and 2018YFF0300606), the National Natural Science Foundation of China (No. 91648207), the Beijing Natural Science Foundation (No. L182001), and the Beijing Municipal Science and Technology Project (No. Z181100009218007).

References

  1. B. E. Lawson, H. A. Varol, A. Huff, E. Erdemir, and M. Goldfarb, “Control of stair ascent and descent with a powered transfemoral prosthesis,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 21, no. 3, pp. 466–473, 2013. View at: Publisher Site | Google Scholar
  2. J. Zhu, Q. Wang, and L. Wang, “On the design of a powered transtibial prosthesis with stiffness adaptable ankle and toe joints,” IEEE Transactions on Industrial Electronics, vol. 61, no. 9, pp. 4797–4807, 2014. View at: Publisher Site | Google Scholar
  3. S. K. Au and H. Herr, “Powered ankle-foot prosthesis,” IEEE Robotics and Automation Magazine, vol. 15, no. 3, pp. 52–59, 2008. View at: Publisher Site | Google Scholar
  4. H. L. Bartlett, B. Lawson, and M. Goldfarb, “Design, control, and preliminary assessment of a multifunctional semi-powered ankle prosthesis,” IEEE/ASME Transactions on Mechatronics, vol. 24, no. 4, pp. 1532–1540, 2019. View at: Publisher Site | Google Scholar
  5. Q. Wang, K. Yuan, J. Zhu, and L. Wang, “Walk the walk: a lightweight active transtibial prosthesis,” IEEE Robotics and Automation Magazine, vol. 22, no. 4, pp. 80–89, 2015. View at: Publisher Site | Google Scholar
  6. S. K. Au, J. Weber, and H. Herr, “Powered ankle–foot prosthesis improves walking metabolic economy,” IEEE Transactions on Robotics, vol. 25, no. 1, pp. 51–66, 2009. View at: Publisher Site | Google Scholar
  7. Y. Feng and Q. Wang, “Combining push-off power and nonlinear damping behaviors for a lightweight motor-driven transtibial prosthesis,” IEEE/ASME Transactions on Mechatronics, vol. 22, no. 6, pp. 2512–2523, 2017. View at: Publisher Site | Google Scholar
  8. D. A. Winter and S. E. Sienko, “Biomechanics of below-knee amputee gait,” Journal of Biomechanics, vol. 21, no. 5, pp. 361–367, 1988. View at: Publisher Site | Google Scholar
  9. R. Robert, R. Marco, and F. Carlo, “Stair ascent and descent at different inclinations,” Gait & Posture, vol. 15, no. 1, pp. 32–44, 2002. View at: Publisher Site | Google Scholar
  10. A. S. Mcintosh, K. T. Beatty, L. N. Dwan, and D. R. Vickers, “Gait dynamics on an inclined walkway,” Journal of Biomechanics, vol. 39, no. 13, pp. 2491–2502, 2006. View at: Publisher Site | Google Scholar
  11. E. J. Wolf, V. Q. Everding, A. A. Linberg, J. M. Czerniecki, and J. M. Gambel, “Comparison of the power knee and C-leg during step-up and sit-to-stand tasks,” Gait & Posture, vol. 38, no. 3, pp. 397–402, 2013. View at: Publisher Site | Google Scholar
  12. H. Huang, F. Zhang, L. J. Hargrove, Z. Dou, and K. B. Englehart, “Continuous locomotion-mode identification for prosthetic legs based on neuromuscular-mechanical fusion,” IEEE Transactions on Biomedical Engineering, vol. 58, no. 10, pp. 2867–2875, 2011. View at: Publisher Site | Google Scholar
  13. E. Zheng and Q. Wang, “Noncontact capacitive sensing-based locomotion transition recognition for amputees with robotic transtibial prostheses,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 2, pp. 161–170, 2017. View at: Publisher Site | Google Scholar
  14. H. L. Bartlett and M. Goldfarb, “A phase variable approach for IMU-based locomotion activity recognition,” IEEE Transactions on Biomedical Engineering, vol. 65, no. 6, pp. 1330–1338, 2018. View at: Publisher Site | Google Scholar
  15. D. Xu, Y. Feng, J. Mai, and Q. Wang, “Real-time on-board recognition of continuous locomotion modes for amputees with robotic transtibial prostheses,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 26, no. 10, pp. 2015–2025, 2018. View at: Publisher Site | Google Scholar
  16. J. A. Spanias, A. M. Simon, S. B. Finucane, E. J. Perreault, and L. J. Hargrove, “Online adaptive neural control of a robotic lower limb prosthesis,” Journal of Neural Engineering, vol. 15, no. 1, article 016015, 2018. View at: Publisher Site | Google Scholar
  17. N. Thatte, T. Shah, and H. Geyer, “Robust and adaptive lower limb prosthesis stance control via extended Kalman filter-based gait phase estimation,” IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 3129–3136, 2019. View at: Publisher Site | Google Scholar
  18. D. Quintero, D. J. Villarreal, D. J. Lambert, S. Kapp, and R. D. Gregg, “Continuous-phase control of a powered knee-ankle prosthesis: amputee experiments across speeds and inclines,” IEEE Transactions on Robotics, vol. 34, no. 3, pp. 686–701, 2018. View at: Publisher Site | Google Scholar
  19. M. R. Tucker, J. Olivier, A. Pagel et al., “Control strategies for active lower extremity prosthetics and orthotics: a review,” Journal of Neuroengineering and Rehabilitation, vol. 12, no. 1, p. 1, 2015. View at: Publisher Site | Google Scholar
  20. T. Yan, M. Cempini, C. M. Oddo, and N. Vitiello, “Review of assistive strategies in powered lower-limb orthoses and exoskeletons,” Robotics and Autonomous Systems, vol. 64, pp. 120–136, 2015. View at: Publisher Site | Google Scholar
  21. M. Tschiedel, M. Russold, and E. Kaniusas, “Relying on more sense for enhancing lower limb prostheses control: a review,” Journal of NeuroEngineering and Rehabilitation, vol. 17, no. 1, p. 99, 2020. View at: Publisher Site | Google Scholar
  22. D. Novak and R. Riener, “A survey of sensor fusion methods in wearable robotics,” Robotics and Autonomous Systems, vol. 73, pp. 155–170, 2015. View at: Publisher Site | Google Scholar
  23. M. Windrich, M. Grimmer, O. Christ, S. Rinderknecht, and P. Beckerle, “Active lower limb prosthetics: a systematic review of design issues and solutions,” Biomedical Engineering Online, vol. 15, no. S3, article 140, 2016. View at: Publisher Site | Google Scholar
  24. D. Xu and Q. Wang, “On-board training strategy for IMU-based real-time locomotion recognition of transtibial amputees with robotic prostheses,” Frontiers in Neurorobotics, vol. 14, no. 47, pp. 1–12, 2020. View at: Publisher Site | Google Scholar
  25. A. J. Young and L. J. Hargrove, “A classification method for user-independent intent recognition for transfemoral amputees using powered lower limb prostheses,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 24, no. 2, pp. 217–225, 2016. View at: Publisher Site | Google Scholar
  26. A. J. Young, A. M. Simon, and L. J. Hargrove, “A training method for locomotion mode prediction using powered lower limb prostheses,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 22, no. 3, pp. 671–677, 2014. View at: Publisher Site | Google Scholar
  27. J. Mendez, S. Hood, A. Gunnel, and T. Lenzi, “Powered knee and ankle prosthesis with indirect volitional swing control enables level-ground walking and crossing over obstacles,” Science Robotics, vol. 5, no. 44, article eaba6635, 2020. View at: Publisher Site | Google Scholar
  28. C. Pew and G. K. Klute, “Turn intent detection for control of a lower limb prosthesis,” IEEE Transactions on Biomedical Engineering, vol. 65, no. 4, pp. 789–796, 2018. View at: Publisher Site | Google Scholar
  29. K. Yuan, Q. Wang, and L. Wang, “Fuzzy-logic-based terrain identification with multisensor fusion for transtibial amputees,” IEEE/ASME Transactions on Mechatronics, vol. 20, no. 2, pp. 618–630, 2015. View at: Publisher Site | Google Scholar
  30. F. Sup, A. Bohara, and M. Goldfarb, “Design and control of a powered transfemoral prosthesis,” International Journal of Robotics Research, vol. 27, no. 2, pp. 263–273, 2008. View at: Publisher Site | Google Scholar
  31. F. Sup, H. A. Varol, and M. Goldfarb, “Upslope walking with a powered knee and ankle prosthesis: initial results with an amputee subject,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 19, no. 1, pp. 71–78, 2011. View at: Publisher Site | Google Scholar
  32. F. Zhang, M. Liu, and H. Huang, “Effects of locomotion mode recognition errors on volitional control of powered above-knee prostheses,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 23, no. 1, pp. 64–72, 2015. View at: Publisher Site | Google Scholar
  33. D. J. Villarreal, H. A. Poonawala, and R. D. Gregg, “A robust parameterization of human gait patterns across phase-shifting perturbations,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 3, pp. 265–278, 2017. View at: Publisher Site | Google Scholar
  34. Y. D. Li and E. T. Hsiao-Wecksler, “Gait mode recognition and control for a portable-powered ankle-foot orthosis,” in 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR), pp. 1–8, Seattle, USA, 2013. View at: Publisher Site | Google Scholar
  35. E. Zheng, L. Wang, K. Wei, and Q. Wang, “A noncontact capacitive sensing system for recognizing locomotion modes of transtibial amputees,” IEEE Transactions on Biomedical Engineering, vol. 61, no. 12, pp. 2911–2920, 2014. View at: Publisher Site | Google Scholar
  36. B. Chen, E. Zheng, X. Fan, T. Liang, and Q. Wang, “Locomotion mode classification using a wearable capacitive sensing system,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 21, no. 5, pp. 744–755, 2013. View at: Publisher Site | Google Scholar
  37. H. Huang, T. A. Kuiken, and R. D. Lipschutz, “A strategy for identifying locomotion modes using surface electromyography,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 1, pp. 65–73, 2009. View at: Publisher Site | Google Scholar
  38. A. K. Godiyal, M. Mondal, S. D. Joshi, and D. Joshi, “Force myography based novel strategy for locomotion classification,” IEEE Transactions on Human-Machine Systems, vol. 48, no. 6, pp. 648–657, 2018. View at: Publisher Site | Google Scholar
  39. H. A. Varol, F. Sup, and M. Goldfarb, “Multiclass real-time intent recognition of a powered lower limb prosthesis,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 3, pp. 542–551, 2010. View at: Publisher Site | Google Scholar
  40. S. Au, M. Berniker, and H. Herr, “Powered ankle-foot prosthesis to assist level-ground and stair-descent gaits,” Neural Networks, vol. 21, no. 4, pp. 654–666, 2008. View at: Publisher Site | Google Scholar
  41. B. Chen, E. Zheng, Q. Wang, and L. Wang, “A new strategy for parameter optimization to improve phase-dependent locomotion mode recognition,” Neurocomputing, vol. 149, pp. 585–593, 2015. View at: Publisher Site | Google Scholar
  42. E. Zheng, S. Manca, T. Yan, A. Parri, N. Vitiello, and Q. Wang, “Gait phase estimation based on noncontact capacitive sensing and adaptive oscillators,” IEEE Transactions on Biomedical Engineering, vol. 64, no. 10, pp. 2419–2430, 2017. View at: Publisher Site | Google Scholar
  43. J. Chen, X. Zhang, Y. Cheng, and N. Xi, “Surface EMG based continuous estimation of human lower limb joint angles by using deep belief networks,” Biomedical Signal Processing and Control, vol. 40, pp. 335–342, 2018. View at: Publisher Site | Google Scholar
  44. A. J. Young, A. M. Simon, N. P. Fey, and L. J. Hargrove, “Classifying the intent of novel users during human locomotion using powered lower limb prostheses,” in 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 311–314, San Diego, USA, 2013. View at: Publisher Site | Google Scholar
  45. S. H. Scott, “Optimal feedback control and the neural basis of volitional motor control,” Nature Reviews Neuroscience, vol. 5, no. 7, pp. 532–545, 2004. View at: Publisher Site | Google Scholar
  46. J. Shi, Y. P. Zheng, Q. H. Huang, and X. Chen, “Continuous monitoring of sonomyography, electromyography and torque generated by normal upper arm muscles during isometric contraction: sonomyography assessment for arm muscles,” IEEE Transactions on Biomedical Engineering, vol. 55, no. 3, pp. 1191–1198, 2008. View at: Publisher Site | Google Scholar
  47. H. Huang, F. Zhang, Y. Sun, and H. He, “Design of a robust EMG sensing interface for pattern classification,” Journal of Neural Engineering, vol. 7, no. 5, article 056005, 2010. View at: Publisher Site | Google Scholar
  48. D. H. Kim, C. Y. Cho, and J. H. Ryu, “Real-time locomotion mode recognition employing correlation feature analysis using EMG pattern,” ETRI Journal, vol. 36, no. 1, pp. 99–105, 2014. View at: Publisher Site | Google Scholar
  49. R. Gupta and R. Agarwal, “Continuous human locomotion identification for lower limb prosthesis control,” CSI Transactions on ICT, vol. 6, no. 1, pp. 17–31, 2018. View at: Publisher Site | Google Scholar
  50. B. Chen, Q. Wang, and L. Wang, “Adaptive slope walking with a robotic transtibial prosthesis based on volitional EMG control,” IEEE/ASME Transactions on Mechatronics, vol. 20, no. 5, pp. 2146–2157, 2015. View at: Publisher Site | Google Scholar
  51. D. Joshi, B. H. Nakamura, and M. E. Hahn, “High energy spectrogram with integrated prior knowledge for EMG-based locomotion classification,” Medical Engineering & Physics, vol. 37, no. 5, pp. 518–524, 2015. View at: Publisher Site | Google Scholar
  52. M. Z. Jamal, “Signal acquisition using surface EMG and circuit design considerations for robotic prosthesis,” in Computational Intelligence in Electromyography Analysis-A Perspective on Current Applications and Future Challenges, G. R. Naik, Ed., INTECH Open Access Publisher, 2012. View at: Publisher Site | Google Scholar
  53. J. Rafiee, M. A. Rafiee, F. Yavari, and M. P. Schoen, “Feature extraction of forearm EMG signals for prosthetics,” Expert Systems with Applications, vol. 38, no. 4, pp. 4058–4067, 2011. View at: Publisher Site | Google Scholar
  54. A. Phinyomark, F. Quaine, S. Charbonnier, C. Serviere, F. Tarpin-Bernard, and Y. Laurillau, “EMG feature evaluation for improving myoelectric pattern recognition robustness,” Expert Systems with Applications, vol. 40, no. 12, pp. 4832–4840, 2013. View at: Publisher Site | Google Scholar
  55. C. D. Hoover, G. D. Fulk, and K. B. Fite, “The design and initial experimental validation of an active myoelectric transfemoral prosthesis,” Journal of Medical Devices, vol. 6, no. 1, article 011005, 2012. View at: Publisher Site | Google Scholar
  56. Y. Feng and Q. Wang, “Using one strain gauge bridge to detect gait events for a robotic prosthesis,” Robotica, vol. 37, no. 11, pp. 1987–1997, 2019. View at: Publisher Site | Google Scholar
  57. Y. Zhang, G. Pan, K. Jia, M. Lu, Y. Wang, and Z. Wu, “Accelerometer-based gait recognition by sparse representation of signature points with clusters,” IEEE Transactions on Cybernetics, vol. 45, no. 9, pp. 1864–1875, 2015. View at: Publisher Site | Google Scholar
  58. Y. Feng, W. Chen, and Q. Wang, “A strain gauge based locomotion mode recognition method using convolutional neural network,” Advanced Robotics, vol. 33, no. 5, pp. 254–263, 2019. View at: Publisher Site | Google Scholar
  59. N. Ahmad, R. A. R. Ghazilla, N. M. Khairi, and V. Kasi, “Reviews on various inertial measurement unit (IMU) sensor applications,” International Journal of Signal Processing Systems, vol. 1, no. 2, pp. 256–262, 2013. View at: Publisher Site | Google Scholar
  60. U. Martinez-Hernandez and A. A. Dehghani-Sanij, “Adaptive Bayesian inference system for recognition of walking activities and prediction of gait events using wearable sensors,” Neural Networks, vol. 102, pp. 107–119, 2018. View at: Publisher Site | Google Scholar
  61. P. R. Cavanagh and P. V. Komi, “Electromechanical delay in human skeletal muscle under concentric and eccentric contractions,” European Journal of Applied Physiology and Occupational Physiology, vol. 42, no. 3, pp. 159–163, 1979. View at: Publisher Site | Google Scholar
  62. L. Tian, B. Zimmerman, A. Akhtar et al., “Large-area MRI-compatible epidermal electronic interfaces for prosthetic control and cognitive monitoring,” Nature Biomedical Engineering, vol. 3, no. 3, pp. 194–205, 2019. View at: Publisher Site | Google Scholar
  63. K. De Pauw, P. Cherelle, B. Tassignon et al., “Cognitive performance and brain dynamics during walking with a novel bionic foot: a pilot study,” PLoS One, vol. 14, no. 4, article e0214711, 2019. View at: Publisher Site | Google Scholar
  64. H. Gao, L. Luo, M. Pi et al., “EEG-based volitional control of prosthetic legs for walking in different terrains,” IEEE Transactions on Automation Science and Engineering, vol. 18, no. 2, pp. 530–540, 2021. View at: Publisher Site | Google Scholar
  65. A. J. Young, T. A. Kuiken, and L. J. Hargrove, “Analysis of using EMG and mechanical sensors to enhance intent recognition in powered lower limb prostheses,” Journal of Neural Engineering, vol. 11, no. 5, article 056021, 2014. View at: Publisher Site | Google Scholar
  66. D. C. Tkach and L. J. Hargrove, “Neuromechanical sensor fusion yields highest accuracies in predicting ambulation mode transitions for transtibial amputees,” in 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3074–3077, Osaka, Japan, 2013. View at: Publisher Site | Google Scholar
  67. P. Zhou, B. Lock, and T. A. Kuiken, “Real time ECG artifact removal for myoelectric prosthesis control,” Physiological Measurement, vol. 28, no. 4, pp. 397–413, 2007. View at: Publisher Site | Google Scholar
  68. L. Hargrove, P. Zhou, K. Englehart, and T. A. Kuiken, “The effect of ECG interference on pattern-recognition-based myoelectric control for targeted muscle reinnervated patients,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 9, pp. 2197–2201, 2009. View at: Publisher Site | Google Scholar
  69. H. J. Hermens, B. Frerisk, R. Merletti et al., “European recommendations for surface electromyography: results of the SENIAM project,” Roessingh Research and Development, vol. 8, pp. 13–54, 1999. View at: Google Scholar
  70. P. R. S. Sanches, A. F. Mller, L. Carro, A. A. Susin, and P. Nohama, “Analog reconfigurable technologies for EMG signal processing,” Revista brasileira de engenharia biomdica. Rio de Janeiro, RJ, vol. 23, no. 2, pp. 153–157, 2007. View at: Google Scholar
  71. G. Li, A. E. Schultz, and T. A. Kuiken, “Quantifying pattern recognition-based myoelectric control of multifunctional transradial prostheses,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 18, no. 2, pp. 185–192, 2010. View at: Publisher Site | Google Scholar
  72. K. Englehart and B. Hudgins, “A robust, real-time control scheme for multifunction myoelectric control,” IEEE Transactions on Biomedical Engineering, vol. 50, no. 7, pp. 848–854, 2003. View at: Publisher Site | Google Scholar
  73. M. Liu, F. Zhang, and H. Huang, “An adaptive classification strategy for reliable locomotion mode recognition,” Sensors, vol. 17, no. 9, p. 2020, 2017. View at: Publisher Site | Google Scholar
  74. M. Elhoushi, J. Georgy, A. Noureldin, and M. J. Korenberg, “A survey on approaches of motion mode recognition using sensors,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 7, pp. 1662–1686, 2017. View at: Publisher Site | Google Scholar
  75. B. Hudgins, P. Parker, and R. N. Scott, “A new strategy for multifunction myoelectric control,” IEEE Transactions on Biomedical Engineering, vol. 40, no. 1, pp. 82–94, 1993. View at: Publisher Site | Google Scholar
  76. K. Englehart, B. Hudgins, P. A. Parker, and M. Stevenson, “Classification of the myoelectric signal using time-frequency based representations,” Medical Engineering & Physics, vol. 21, no. 6-7, pp. 431–438, 1999. View at: Publisher Site | Google Scholar
  77. D. Graupe, J. Salahi, and K. H. Kohn, “Multifunctional prosthesis and orthosis control via microcomputer identification of temporal pattern differences in single-site myoelectric signals,” Journal of Biomedical Engineering, vol. 4, no. 1, pp. 17–22, 1982. View at: Publisher Site | Google Scholar
  78. D. Tkach, H. Huang, and T. A. Kuiken, “Study of stability of time-domain features for electromyographic pattern recognition,” Journal of Neuroengineering and Rehabilitation, vol. 7, no. 1, p. 21, 2010. View at: Publisher Site | Google Scholar
  79. F. Zhang and H. Huang, “Source selection for real-time user intent recognition toward volitional control of artificial legs,” IEEE Journal of Biomedical and Health Informatics, vol. 17, no. 5, pp. 907–914, 2013. View at: Publisher Site | Google Scholar
  80. C. M. Bishop, Pattern recognition and machine learning, Springer Science Business Media, New York, USA, 2006.
  81. L. J. Hargrove, A. M. Simon, A. J. Young et al., “Robotic leg control with EMG decoding in an amputee with nerve transfers,” New England Journal of Medicine, vol. 369, no. 13, pp. 1237–1242, 2013. View at: Publisher Site | Google Scholar
  82. A. J. Young, A. M. Simon, N. P. Fey, and L. J. Hargrove, “Intent recognition in a powered lower limb prosthesis using time history information,” Annals of Biomedical Engineering, vol. 42, no. 3, pp. 631–641, 2014. View at: Publisher Site | Google Scholar
  83. K. H. Ha, H. A. Varol, and M. Goldfarb, “Volitional control of a prosthetic knee using surface electromyography,” IEEE Transactions on Biomedical Engineering, vol. 58, no. 1, pp. 144–151, 2011. View at: Publisher Site | Google Scholar
  84. J. Mai, D. Xu, H. Li, S. Zhang, J. Tan, and Q. Wang, “Implementing a SoC-FPGA based acceleration system for on-board SVM training for robotic transtibial prostheses,” in 2018 IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 150–155, Kandima, Maldives, 2018. View at: Publisher Site | Google Scholar
  85. J. Mai, W. Chen, S. Zhang, D. Xu, and Q. Wang, “Performance analysis of hardware acceleration for locomotion mode recognition in robotic prosthetic control,” in 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS), pp. 607–611, Shenzhen, China, 2018. View at: Publisher Site | Google Scholar
  86. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, Wiley-Interscience, New York, USA, 2001.
  87. D. Novak, M. Mihelj, and M. Munih, “A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing,” Interacting with Computers, vol. 24, no. 3, pp. 154–172, 2012. View at: Publisher Site | Google Scholar
  88. P. Geethanjali and K. K. Ray, “Identification of motion from multi-channel EMG signals for control of prosthetic hand,” Australasian Physical & Engineering Sciences in Medicine, vol. 34, no. 3, pp. 419–427, 2011. View at: Publisher Site | Google Scholar
  89. E. J. Scheme, K. B. Englehart, and B. S. Hudgins, “Selective classification for improved robustness of myoelectric control under nonideal conditions,” IEEE Transactions on Biomedical Engineering, vol. 58, no. 6, pp. 1698–1705, 2011. View at: Publisher Site | Google Scholar
  90. I. Guyon and A. Elisseeff, “An introduction to variable and feature selection,” Journal of Machine Learning Research, vol. 3, no. 6, pp. 1157–1182, 2003. View at: Google Scholar
  91. K. S. Fu, Sequential Methods in Pattern Recognition and Machine Learning, Academic Press, New York, USA, 2007.
  92. D. W. Aha and R. L. Bankert, “A comparative evaluation of sequential feature selection algorithms,” in Learning from Data, D. Fisher and H. J. Lenz, Eds., Lecture Notes in Statistics, pp. 199–206, Springer, New York, USA, 1996. View at: Publisher Site | Google Scholar
  93. A. Kusiak, “Rough set theory: a data mining tool for semiconductor manufacturing,” IEEE Transactions on Electronics Packaging Manufacturing, vol. 24, no. 1, pp. 44–50, 2001. View at: Publisher Site | Google Scholar
  94. K. Englehart, B. Hudgins, and P. A. Parker, “A wavelet-based continuous classification scheme for multifunction myoelectric control,” IEEE Transactions on Biomedical Engineering, vol. 48, no. 3, pp. 302–311, 2001. View at: Publisher Site | Google Scholar
  95. D. Yuan, Y. Liang, L. Yi, Q. Xu, and O. M. Kvalheim, “Uncorrelated linear discriminant analysis (ULDA): a powerful tool for exploration of metabolomics data,” Chemometrics and Intelligent Laboratory Systems, vol. 93, no. 1, pp. 70–79, 2008. View at: Publisher Site | Google Scholar
  96. N. F. Gler and S. Kocer, “Use of support vector machines and neural network in diagnosis of neuromuscular disorders,” Journal of Medical Systems, vol. 29, no. 3, pp. 271–284, 2005. View at: Publisher Site | Google Scholar
  97. L. J. Hargrove, G. Li, K. B. Englehart, and B. S. Hudgins, “Principal components analysis preprocessing for improved classification accuracies in pattern-recognition-based myoelectric control,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 5, pp. 1407–1414, 2009. View at: Publisher Site | Google Scholar
  98. B. J. Frey, Graphical Models for Machine Learning and Digital Communication, MIT press, Cambridge, USA, 1998. View at: Publisher Site
  99. J. W. Sensinger, B. A. Lock, and T. A. Kuiken, “Adaptive pattern recognition of myoelectric signals: exploration of conceptual framework and practical algorithms,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 17, no. 3, pp. 270–278, 2009. View at: Publisher Site | Google Scholar
  100. T. Matsubara, S. H. Hyon, and J. Morimoto, “Learning and adaptation of a stylistic myoelectric interface: EMG-based robotic control with individual user differences,” in 2011 IEEE International Conference on Robotics and Biomimetics, pp. 390–395, Karon Beach, Phuket, Thailand, 2011. View at: Publisher Site | Google Scholar
  101. T. Tommasi, F. Orabona, C. Castellini, and B. Caputo, “Improving control of dexterous hand prostheses using adaptive learning,” IEEE Transactions on Robotics, vol. 29, no. 1, pp. 207–219, 2013. View at: Publisher Site | Google Scholar
  102. L. Du, F. Zhang, H. He, and H. Huang, “Improving the performance of a neural-machine interface for prosthetic legs using adaptive pattern classifiers,” in 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1571–1574, Osaka, Japan, 2013. View at: Publisher Site | Google Scholar
  103. J. A. Spanias, E. J. Perreault, and L. J. Hargrove, “A strategy for labeling data for the neural adaptation of a powered lower limb prosthesis,” in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 3090–3093, Chicago, USA, 2014. View at: Publisher Site | Google Scholar
  104. J. A. Spanias, A. M. Simon, E. J. Perreault, and L. J. Hargrove, “Preliminary results for an adaptive pattern recognition system for novel users using a powered lower limb prosthesis,” in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5083–5086, Orlando, USA, 2016. View at: Publisher Site | Google Scholar
  105. J. A. Spanias, A. M. Simon, and L. J. Hargrove, “Across-user adaptation for a powered lower limb prosthesis,” in 2017 International Conference on Rehabilitation Robotics (ICORR), pp. 1580–1583, London, UK, 2017. View at: Publisher Site | Google Scholar
  106. X. Zhang, Y. Liu, F. Zhang et al., “On design and implementation of neural-machine interface for artificial legs,” IEEE Transactions on Industrial Informatics, vol. 8, no. 2, pp. 418–429, 2012. View at: Publisher Site | Google Scholar
  107. A. D. C. Chan and K. B. Englehart, “Continuous myoelectric control for powered prostheses using hidden Markov models,” IEEE Transactions on Biomedical Engineering, vol. 52, no. 1, pp. 121–124, 2005. View at: Publisher Site | Google Scholar
  108. A. M. Simon, L. J. Hargrove, B. A. Lock, and T. A. Kuiken, “A decision-based velocity ramp for minimizing the effect of misclassifications during real-time pattern recognition control,” IEEE Transactions on Biomedical Engineering, vol. 58, no. 8, pp. 2360–2368, 2011. View at: Publisher Site | Google Scholar
  109. B. Chen, E. Zheng, and Q. Wang, “A locomotion intent prediction system based on multi-sensor fusion,” Sensors, vol. 14, no. 7, pp. 12349–12369, 2014. View at: Publisher Site | Google Scholar
  110. D. E. Ledoux, “Inertial sensing for gait event detection and transfemoral prosthesis control strategy,” IEEE Transactions on Biomedical Engineering, vol. 65, no. 12, pp. 2704–2712, 2018. View at: Publisher Site | Google Scholar
  111. D. Xu, S. Crea, N. Vitiello, and Q. Wang, “Capacitive sensing-based continuous gait phase estimation in robotic transtibial prostheses,” in 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), pp. 298–303, New York, USA, 2020. View at: Publisher Site | Google Scholar
  112. D. Xu, Y. Yang, R. Yang, and Q. Wang, “Maximum dorsiflexion detection based on an on-board adaptive algorithm for transtibial amputees with robotic prostheses,” IEEE Transactions on Automation Science and Engineering, vol. 18, no. 2, pp. 437–447, 2021. View at: Publisher Site | Google Scholar
  113. D. Xu, S. Crea, N. Vitiello, and Q. Wang, “Online estimation of continuous gait phase for robotic transtibial prostheses based on adaptive oscillators,” in 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), pp. 1890–1895, Boston, MA, USA, 2020. View at: Publisher Site | Google Scholar
  114. M. A. Holgate, T. G. Sugar, and A. W. Bohler, “A novel control algorithm for wearable robotics using phase plane invariants,” in 2009 IEEE International Conference on Robotics and Automation, pp. 3845–3850, Kobe, Japan, 2009. View at: Publisher Site | Google Scholar
  115. J. W. Jeong, M. K. Kim, H. Cheng et al., “Capacitive epidermal electronics for electrically safe, long-term electrophysiological measurements,” Advanced Healthcare Materials, vol. 3, no. 5, pp. 642–648, 2014. View at: Publisher Site | Google Scholar

Copyright © 2021 Dongfang Xu and Qining Wang. Exclusive Licensee Beijing Institute of Technology Press. Distributed under a Creative Commons Attribution License (CC BY 4.0).

 PDF Download Citation Citation
Views353
Downloads242
Altmetric Score
Citations