With the increasing technical advancement and system complexity, modern systems being operated in dynamically environment require constant human supervision although the amount of direct human operation has been reduced considerably. As the matter of facts, the relative share of human workloads, the mental workload rather than physical workload, is increased across all modern applications, especially in the air transportation industry. More and more evidences showed that human errors have become the main reasons to result in aviation disaster [1].

In order to make sure the safety, comfort, and continued productive efficiency of the operator, a reasonable goal is to regulate mental workload so that they neither underload nor overload an individual. The evaluation of mental workload is also an important aspect of human factors research. The operator who bear abnormal workload would hardly maintain performance at required levels [2]. And it is a widely accepted assumption that the level of mental workload has apparent link to the human operational performance, especially the potential occurrence of unsafe operation of the system.

It was also realized that the performance of a human operator could be affected by the complexity of a particular task as well as various factors of the human operator during the operation of the system such as fatigue, stress, mental workload, attention deficit, and executive function, among others, which leads to errors, accidents, or even disasters. Moreover, the external characteristics of operator’s workload can be measured using physiological parameters in real time. Researchers have been investigated for decades to try to find optimal combination to represent workloads accurately and then to predict the performance of the operators.

In this paper, the research was based on the assumption that the main electromyography (EMG) signals patterns for a particular task should be remain similar since it reflects natural requirements relating to the motor action associated with the task, and the secondary signal patterns associated with the main patterns should be evolving when the operator repeat the same task reflecting the operator’s capabilities improvement. The content of our research includes the establishment of the experiment rig, designing and carrying out tasks, and analyzing the results. And first of all, the mapping between the state of hand motions and Surface Electromyography (sEMG) should be determined.

The EMG signal represents the electrical activity of muscles. EMG signals are usually detected via surface electrodes attached on the skin. EMG measures electrical currents that are generated in a muscle during its contraction. EMG signals can be used for a variety of applications including clinical applications, human-computer interaction and interactive computer gaming [3,4]. Moreover, EMG can be used to sense isometric muscular activity which does not translate into movement. This makes it possible to classify subtle motionless gestures and to control interfaces without being noticed and without disrupting the surrounding environment [5].

In order to obtain the signals regarding hand motion states, the sensors were placed on the right arm of the objects. The experiment about the sEMG acquisition of postures should be done to get the effects imposed by hand status. Taking into account the needs of experimental tasks, the directions of hand movements are divided into four categories, they are left, right, forward, and backward, respectively. The signal acquisition is separated into two phases: gesture acquisition and motion acquisition.

An sEMG gesture acquisition experiment should be carried out for the purpose of verifying the effect induced by the movement of arm on the sEMG signal characteristics during the posture retention of the arm muscles. And it is named static posture acquisition. Static posture acquisition is to explore the differentiation of the sEMG signal of particular muscles that maintain different postures of hand movements.

The experiment of motion data acquisition should be done to obtain the sEMG signals in the period of uniform motion caused by the right hand. The purpose of the test was to reveal the ability of a particular muscle EMG to recognize the periodic motion of the arm.

2.1Participants

Ten volunteers (5 males and 5 females) were participated in this experiment with the age range from 22 to 26. They are all the postgraduates from School of Electronic Information and Electrical Engineering. Subjects have healthy body, normal eyesight or correction normal eyesight. Before the experiment, they were ensured to have adequate sleep without intense exercise. All of them were completely understood what would be done in the experiment and signed the consent form. The arm of the subject has no history of major injuries. Within 24 h before the experiment they did not take any irritating items.

2.2Apparatus

Two kinds of devices were used throughout the experiment. The one is flying joystick (named Extreme 3D Pro) [6] attached Logitech trademark. The perfect ergonomic design with a custom twist-handle rudder relies its one-handed control resulting in a smaller device footprint. There are six programmable buttons on the base. Each programmable button can be configured to execute simple single commands or intricate macros involving multiple keystrokes, mouse events, and more. In this experiment, we only operate the rocker in the above four directions. (see Fig. 1)

Myoelectric device (left) and flying joystick (right)

Full size image

The other device is Delysis Trigno^TM Wireless System [7], it was used to record the sEMG of the right forearm during operating the flying joystick. With a guaranteed transmission range of 40 m, a rechargeable battery lasting up to 8 h, and an intelligent sensor design, three-axis accelerometer is embedded in each EMG sensor. And the 64-channel synchronous signals can be outputted at the same time. Moreover, it can provide broader analysis data types. The high-frequency cutoff frequency of the amplifier is 5000 Hz, and the low-frequency cutoff frequency is 10 Hz. The range of signal amplitude of sEMG from 0mv to 10 mv, and its frequency ranges from 20–500 Hz [5]. The sEMG digitized by A/D converter at a sampling frequency of 2000 Hz in this experiment. The signals were amplified by a factor of 30 (see Fig. 1).

2.3Muscle Selection and Electrode Placement

When selecting the muscle as the source of sEMG signals, the following four aspects should be taken into account [8]: the function of the muscle should be directly related to the flexion and extension of the arm joint movement, the shape of the muscle should be relatively large enough, the muscle’s position should be located in the shallow layer of their muscles, and the reliability of the acquisition results should be guaranteed when collecting the sEMG of the selected muscle.

According to the principle of local anatomy, the movement to bend and stretch the arm is mainly controlled by the upper arm muscle zone, holding fist and extensor are mainly controlled by the forearm muscle group. In this experiment, the flexor carpi ulnaris, flexor carpi radialis, brachioradialis, biceps brachii and palmaris longus are selected as the source of the sEMG (see Fig. 2).

Arm muscle anatomy

Full size image

In addition, pay attention to the following points when placing the surface EMG electrodes: Surface electrodes should be placed in the place that is hardly affected by the crosstalk of adjacent muscle. For the majority of superficial muscles, the detection electrode should be placed in the middle of the tested muscle where is far enough from the other muscles. The direction of the arrow on the electrode should be parallel to the direction of the muscle spindles. Avoid placing the electrode on the tendon or near the tendon, avoid getting it close to muscle movement points and placing the electrode on the outer edge of the tested muscle is not recommended. To reduce the electrical impedance between the skin and the electrode, the forearm was shaved and cleaned with Ether and the electrodes were placed according to the anatomy showed on Fig. 2.

2.4Procedure

Throughout the experiment, the subject sat on the ergonomic chair, keeping his/her upper body upright. Ergonomic seat is a kind of simplification of the cockpit seat, to maintain the basic functions. The angle of the backrest and the height of the seat are adjustable. In the experiment, the seat position is fixed and the subject kept the position and postures unchanged.

During the experiment of static posture acquisition, the subject grasped the joystick with his/her right hand, respectively, to the left, right, forward and backward, and recorded sEMG signals corresponds to four kinds of positions. The subjects were required to perform gestures in a sufficiently constant manner to eliminate the interference induced by the difference of postures. Subjects would have some rest between any two actions to excluding the effect caused by muscle fatigue on sEMG signals.

As for the motion data acquisition. Subjects respectively completed the hand movements with exercise cycle of 2 s and 4 s during the experiment for five times. Moreover, the subject also need to complete the sEMG signal acquisition experiment with variable speed exercise in order to show the ability of the sEMG originated particular muscle to recognize the movement of the arm in a periodic jump, and in this case, the period of hand movement varied from 2 s to 4 s. Just as the process of static posture acquisition, muscle fatigue and subjects’ postures should be taken into account.

2.5Data Process

Signal Preprocessing.

The sEMG signal is a complex physiological signal collected by placing the electrode on the muscle surface of the human body. The recorded electrical signal is preamplified and converted by A/D module and then sent to a signal processing module at the back end. The processed result reflects that the human body Muscle activity. The sEMG signal’s energy is mainly concentrated in the following 1000 Hz [5]. From the Shannon sampling theorem, we can see that if we want to recover the sampled signal into the original signal and without distortion, the sampling frequency used should be greater than twice the maximum frequency of the original signal, so in this paper, 2000 Hz signal sampling rate was applied.

The sEMG signals show a train of motor unit action potentials corrupted with noise. Surface EMG signals, like most of the electrophysiological measurements, are frequently corrupted with three categories of noise [9], i.e. power line interference, white Gaussian noise, and motion artifact or baseline wandering. Noise contamination may compromise the efficacy of the EMG reduce the noise from surface EMG signals, among which the most simple and cost-efficient solution is to use conventional digital filters. For instance, the simplest method of removing narrow bandwidth interference from recorded signals is to use a linear recursive digital notch filter. Taking into account the efficiency of the algorithm and the actual needs of this article, the band-pass filter was used.

Activity Segment Detection.

To explore the possibility of hand-motion encoding using multichannel EMG signals, signal-related segments of motion should be detected. During the execution of gesture actions, the sEMG signal detected by the sensors is called the active segment. Active segment can be straightforward to describe the sEMG signal for each action, it can be regarded as a gesture sEMG signal samples. To realize the recognition of gesture actions, it is necessary to detect the active segments of each gesture action from the continuous signals, that is, to determine the start and the end of each gesture action. The existing sEMG activity segment extraction algorithms include moving average method, short-time Fourier method, entropy theory method, etc. [10,11,12].

Considering the efficiency of the algorithm, in this paper, we use the moving average method based on threshold decision to detect the sEMG signals of the gesture actions. The moving average method uses a certain analysis window to calculate the timing signal, and applies the average of the window signal to represent this window signal, and the analysis window with time can predict the future signal direction. Active segment detection judgment is made based on whether the energy of the sEMG signal sequence exceeds a preset threshold. Taking into account the different of the electrode positions caused by sEMG amplitude differences in different channels, active segment detection in all channels sEMG is based on the sum of the average. By selecting the appropriate parameters of the mean square value of each channel moving average, it is convenient to determinate the starting and ending points corresponding to the action. And the specific process is as follows:

1. (a)

   Calculate the summation average of multichannel sEMG signals at time t, and then square the average signal to obtain the instantaneous energy sequence.

   $$ sEMG_{aver} (t) = \left[ {\frac{1}{C}\sum\limits_{k = 1}^{C} {sEMG_{k} (t)} } \right]^{2} \begin{array}{*{20}c} {} & {t < M} \\ \end{array} $$

   (1)

where\( C \) is the number of channels,\( M \) is the total number of signal sampling points.

1. (b)

   Take the width of the active window\( N = 100 \) points (Equivalent to 50 ms signals length at 2000 Hz sample rate), deal with the squared signals movingly by averaging to get the value of moving average at point t.

   $$ sEMG_{MA} (t) = \frac{1}{N}\sum\limits_{n = t}^{t - N + 1} {sEMG_{aver} (n)} \begin{array}{*{20}c} {} & {t \le M + N - 1} \\ \end{array} $$

   (2)
2. (c)

   Compare\( sEMG_{MA} (t) \) and a definite threshold\( TH \) to determine the action signal. The signals with whose\( sEMG_{MA} (t) \) is greater than\( TH \) and the length exceeds a certain set value are considered as signal segment, otherwise, the signals are known as noise. Described below using the formula:

   $$ sEMG_{rec} (t) = \left\{ \begin{aligned} sEMG_{MA} (t)\begin{array}{*{20}c} {} & {\begin{array}{*{20}c} {if} & {sEMG_{MA} (t) \ge TH} \\ \end{array} } \\ \end{array} \hfill \\ 0\begin{array}{*{20}c} {\begin{array}{*{20}c} {} & {} \\ \end{array} } & {\begin{array}{*{20}c} {} & {\begin{array}{*{20}c} {} & {\begin{array}{*{20}c} {if} & {sEMG_{MA} (t) \ge TH} \\ \end{array} } \\ \end{array} } \\ \end{array} } \\ \end{array} \hfill \\ \end{aligned} \right. $$

   (3)

where\( sEMG_{rec} (t) \) is the rectified signal at time t.

Sliding Window.

After the raw sEMG signals were processed by band-pass filter, the sliding window was employed to deal with the signals. Explaining in detail, the signal of this section was represent by calculating root mean square (RMS) of the sampling points inside the window which was a certain length of the analysis window slid on the timing signal. The RMS value of the sEMG signal, as an index of the time domain of the EMG signal, represents the instantaneous electric power of the EMG signal and can represent the effective value of the muscle surface discharge. Modern research results show that the RMS waveform is similar to the linear envelope waveform of EMG signal and reflects the amplitude variation characteristics of sEMG signal in the time dimension. Its value is related to the synchronization of motion unit recruitment and excitement rhythm, and depends on the intrinsic relationship between the factors of the muscle load and the physiological processes of the muscle itself. And it is often used to describe the state of muscle activity because of its good real-time performance [14]. Hence, we choose RMS as a parameter to evaluate the degree of dynamic muscle activity. By definition, the formula of RMS is as follows:

where\( sEMG(t) \) is the sample value of the muscle surface signal at the time t,\( T \) is the length of time during a sampling period.

Determining the Range of the Threshold.

After getting the activity section of signals, activity segments would be coding. That is Binarization. Binarizing the sEMG signals is actually finding the best step function to fit the signal curves. Here, the step function we use is as follows:

where\( error \) is on behalf of the difference between the true value and the predicted value.\( N_{1} \) is action mode, and its value is an integer and does not exceed the number of eight.\( Q_{ij} \) is the true value.\( R_{ij} \) is the predicted value. And\( R_{ij} \) can be described by the following formula:

where\( thre \) is the threshold of the binarization process.

In the signal processing software environment, this paper uses the band-pass filter to filter the signal. In the process of data acquisition of static postures, every subject were requested to keep the flying joystick in a constant manner toward four kinds of directions. Only two cases are shown here, the result of operation are respectively opened up Figs. 3 and4. In the following pictures, there are two columns. And the left column is the raw sEMG signals after treating by band-pass filter. The five curves represent brachioradialis (abbreviated as Br), flexor carpi radialis (FCR), flexor carpi ulnaris (FCU), biceps brachii (BB) and palmaris longus (PL) from top to bottom, respectively. And the right column shows the root mean square corresponds to the left signals which is obtained by moving the sliding window.

The result of operation to keep joystick leftward

Full size image

The result of operation to keep joystick rightward

Full size image

Combined with Figs. 3 and4, we can see that five curves are generally stable with negligible minor fluctuations in the case of data acquisition of static postures. As the control group experiment, the Fig. 5 shown the result of keeping the right hand on the joystick without exertion. Observing the three pictures, the amplitude of the biceps brachii is almost unchanged. This result shows that the biceps brachii did not participate in exercise during the experiment. Comparing Fig. 3 with Fig. 4, the amplitude of palmaris longus is approximately equal. The voltage’s amplitudes produced by the other three kinds of muscles are different. Therefore, the sEMG signals is sensitive to changes in movement and the value of RMS can be employed as a kind of feature to identify different hand movements.

The result of operation to keep joystick centre position

Full size image

In the process of data acquisition of motion postures, the subjects manipulated the forearm to do periodic exercise. There were only shown the result of operation to the right and back in Figs. 6 and7. From the two pictures, the true that the biceps brachii did not participate in motions was verification. And the signals show periodicity as the periodic exercise of the hand except for the signal induced by the flexor carpi radialis. The sEMG signals amplitude produced by the same muscles in different exercise modes were different. Hence, we can distinguish different actions by extracting different features. Furthermore, from the Fig. 8, we can know that the width of the action signals varies with the length of the action cycle, and the longer the action period, the wider the envelope of the signal. In summary, sEMG signals obtained by carrying out the method mentioned above have strong ability and sensitivity of recognizing hand movements that get dynamically change.

The result of operation to keep periodic exercise backward

Full size image

The result of operation to keep periodic exercise right

Full size image

The result of operation to keep periodic exercise changeable

Full size image

In addition, the abscissa on the graph represents the number of points sampled, and the length between each two points corresponds to 0.5 ms on the time axis. The vertical axis of the graph is on behalf of sEMG signal amplitude, its unit of measurement is V.

For the activity section of signals, the sliding window was used to obtained the corresponding root mean square. Than we can get the error according to the Eq. (5) (Fig. 9). Here, Ten samples are randomly selected, and each sample includes 8 kinds of states that were described as above. From the Fig. 9, the range of optimal threshold could be obtained. And the value range is\( 1.2 \times 10^{ - 5} \)–\( 1.7 \times 10^{ - 5} \).

The picture about error varied with the threshold

Full size image

For further study the optimal threshold, the accuracy was calculated (shown in Fig. 10). The mode1, mode2, mode3 and mode4 are the state of hand motion in the case of dynamic gesture. And the mode5, mode6, mode7 and mode8 are the state of hand motion in the case of static gesture. The left picture is one result of coding, we call it Pattern one. The right picture is another result of coding called as Pattern two. From the Fig. 10, mode2 (downward) and mode3 (leftward) have the same coding and their coding are robust. As for the specific meaning of the pattern, we can know according to Table 1 as follow. When the value of threshold is close to the left of the abscissa, the accuracy of the Pattern One is higher. And when the value of threshold is close to the right of the abscissa, the accuracy of the Pattern Two is higher.

The relationship between threshold and accuracy

Full size image

Once the binarized threshold has been determined, we can get a series of encoding result according to Eq. (6), and the result of coding are partially displayed in Fig. 11.

The partial result of coding

Full size image

A raw sEMG signal contains more important information regarding the nervous system in useless form. In this paper, an experiment was designed to obtained the data of sEMG in different motion modes. The aim of this paper is to give detailed information about clearing up commonly associated noises and artifacts from sEMG signals, and to explore the relationship between the states of muscles of the forearm and sEMG signals. The result shows that the biceps brachii did not participate in exercise during the experiment and sEMG signals have strong ability and sensitivity of recognizing different hand movements. Further, the binarization method to code the sEMG signals was applied in this paper. The result shows that the choice of threshold has a great influence on the result of signal coding and we can not obtain proper coding only using the parameter of time domain. So, in the next work, suitable features including time domain, frequency domain and time-frequency domain features would be selected to batter establish the correspondence for the specific and real experimental environment and platform mentioned above.