CN111816309B - Rehabilitation training prescription self-adaptive recommendation method and system based on deep reinforcement learning - Google Patents

Abstract

Translated fromChinese

本发明提供基于深度强化学习的康复训练处方自适应推荐方法及系统。方法包括：1)收集患者基本信息和病历信息；2)获取患者不同脑区的脑皮层血氧数据和患者患肢的运动和肌电数据；3)利用脑血氧数据计算患者运动康复训练过程中脑功能评价指标，利用运动数据和肌电数据计算患者运动康复训练过程中运动功能评价指标和肌肉功能评价指标，以动态评估患者的脑功能、运动功能和肌肉功能；4)将步骤3)得到的脑功能、运动功能和肌肉功能评价指标输入预先建立的深度强化学习模型，以训练深度强化学习模型并自动生成康复训练处方；5)将步骤4)生成的康复训练处方反馈给医生和患者进行康复训练。利用该方法和系统，能够实现训练处方的自适应调节。

The invention provides an adaptive recommendation method and system for rehabilitation training prescriptions based on deep reinforcement learning. The methods include: 1) collecting the patient's basic information and medical record information; 2) obtaining the cerebral cortex blood oxygen data of different brain regions of the patient and the motion and electromyography data of the patient's affected limb; 3) using the cerebral blood oxygen data to calculate the patient's exercise rehabilitation training process Midbrain function evaluation index, using exercise data and EMG data to calculate the motor function evaluation index and muscle function evaluation index in the process of exercise rehabilitation training of patients, to dynamically evaluate the patient's brain function, motor function and muscle function; 4) Step 3) The obtained evaluation indexes of brain function, motor function and muscle function are input into a pre-established deep reinforcement learning model to train the deep reinforcement learning model and automatically generate a rehabilitation training prescription; 5) Feed back the rehabilitation training prescription generated in step 4) to doctors and patients Do rehabilitation training. Using the method and system, the adaptive adjustment of the training prescription can be realized.

Description

Rehabilitation training prescription self-adaptive recommendation method and system based on deep reinforcement learning

Technical Field

The invention relates to the field of limb exercise rehabilitation training, in particular to a rehabilitation training prescription self-adaptive recommendation method and system based on deep reinforcement learning.

Background

More than 200 million stroke patients are newly added in China every year, and the trend of increasing year by year is shown, wherein 55-75% of stroke patients show dyskinesia. Meanwhile, the brain function damage caused by cerebral palsy, brain trauma and the like can also cause limb movement dysfunction, and heavy burden is brought to patients and families and society thereof. Rehabilitation training is the most important means for recovering the motor function of a patient. However, no matter the traditional artificial rehabilitation training or the rehabilitation training based on the rehabilitation training robot, the establishment of personalized rehabilitation training prescriptions aiming at different conditions of patients is an important condition for guaranteeing the training effect. But at present, the rehabilitation training prescription can only be made by a doctor according to the evaluation result of a patient, and the rehabilitation training prescription depends on the experience of the doctor to a great extent. Moreover, the functional evaluation of the patient is generally performed only several times of regular evaluations at different rehabilitation stages, so the updating of the training prescription also depends on the evaluation period, the updating of the training prescription may not follow the rehabilitation process of the patient, and the improvement of the rehabilitation efficiency is difficult. The development of artificial intelligence enables the rehabilitation training robot to utilize various sensing information to perform real-time function evaluation in the training process of a patient, the problem of long artificial evaluation period is solved, but a doctor is still required to adjust a prescription according to an evaluation result, and the workload of the doctor is undoubtedly greatly increased by frequently adjusting the prescription. On the other hand, active participation, fatigue degree and the like of a patient in rehabilitation training also have important influence on the training effect, the training efficiency under low active participation degree and fatigue state is often low, and manual adjustment of a training prescription is difficult to achieve timely adjustment according to the state of the patient in a single training process, so that the waste of training and treatment resources is caused to a certain extent.

Disclosure of Invention

Based on the above problems, the present invention aims to provide a rehabilitation training prescription adaptive recommendation method and system based on deep reinforcement learning, which performs real-time evaluation of brain function and motor function according to information of patient such as near infrared brain oxygen, motor, myoelectricity, etc., inputs various medical record information and function evaluation indexes into a pre-established deep reinforcement learning model for learning, and adaptively recommends a rehabilitation training prescription according to the functional state of the patient.

One aspect of the invention provides a rehabilitation training prescription self-adaptive recommendation method based on deep reinforcement learning, wherein the method comprises the following steps:

1) collecting basic information and medical record information of a patient;

2) acquiring cerebral cortex blood oxygen data of different brain areas of a patient by using near-infrared cerebral blood oxygen monitoring equipment, and acquiring motion data and myoelectric data of an affected limb of the patient;

3) calculating to obtain a brain function evaluation index in the exercise rehabilitation training process of the patient by utilizing the cerebral blood oxygen data, and calculating to obtain a motor function evaluation index and a muscle function evaluation index in the exercise rehabilitation training process of the patient by utilizing the motor data and the myoelectric data so as to dynamically evaluate the brain function, the motor function and the muscle function of the patient;

4) inputting the evaluation indexes of the brain function, the motor function and the muscle function obtained in the step 3) into a pre-established deep reinforcement learning model so as to train the deep reinforcement learning model and automatically generate a rehabilitation training prescription;

5) feeding back the rehabilitation training prescription generated in the step 4) to the doctor and the patient for rehabilitation training.

According to one embodiment, the training of the depth-enhanced learning model in the step 4) above includes training the depth-enhanced learning model by using the brain function, motor function and muscle function evaluation indexes as states, using the rehabilitation training prescription as an action, and using the function improvement condition after training by using the current rehabilitation training prescription as a reward according to the basic information and medical history information of the patient, and introducing the prior knowledge in the rehabilitation training prescription knowledge base in the training process to accelerate the training of the learning model.

According to an embodiment, the deep reinforcement learning model in the step 4) above may include a rehabilitation training prescription knowledge base based on a large amount of pre-input patient medical record information, a function evaluation index, and a training prescription issued by a doctor, and the model training is assisted by using prior knowledge in the knowledge base.

According to another embodiment, the deep reinforcement learning model in the above step 4) may perform reinforcement learning with the brain function, motor function, and muscle function evaluation indexes of the patient as states, with a rehabilitation training prescription as an action, and with a function improvement situation after training with the current prescription as a reward.

In one embodiment, the evaluation indexes of brain function, motor function and muscle function generated in the step 3) and the rehabilitation training prescription generated in the step 4) can be added to the knowledge base of the deep reinforcement learning model in real time, and the knowledge base is continuously expanded.

In one embodiment, the brain function evaluation index, the motor function evaluation index and the muscle function evaluation index obtained in the step 3) are input into a trained deep reinforcement learning model, Q values of different types or grades contained in each rehabilitation training prescription are output through model calculation, the prescription items with the highest Q values are combined, and the rehabilitation training prescription is automatically generated, wherein the Q values are numerical representations of the corresponding action advantages and disadvantages.

In another embodiment, the brain blood oxygen parameter data and the myoelectric data obtained in the step 2) above can be further analyzed to obtain the active participation and fatigue degree of the brain and the muscle in the training process of the patient, and the deep reinforcement learning model in the step 4) can adjust the training prescription according to the active participation and fatigue degree of the patient.

According to a preferred embodiment, the active participation degree of the brain and the muscle can be reflected by the activation degree of different brain areas and the amplitude information of different muscle electromyographic signals, and the fatigue degree of the muscle of the patient can be reflected by frequency domain information such as the average power frequency, the median frequency and the like of the electromyographic signals.

According to another preferred embodiment, the training of the deep reinforcement learning model in the step 4) above may include first initializing an experience pool and network weights, inputting medical records and state parameters of each function evaluation index, and if the current state cannot match the feature state in the prior knowledge base, selecting a training prescription according to the selected learning strategy; if the current state can be matched with the characteristic state in the prior knowledge base, a training prescription is comprehensively judged and output according to the prior action Q value and the estimated action Q value, the information is stored in an experience pool, the steps are repeated to train the network, the network weight is automatically updated after each training to correct the network, and the selected learning strategy is an epsilon-greedy strategy.

According to another embodiment, the evaluation index generated in the step 3) and the rehabilitation training prescription generated in the step 4) can be added to the knowledge base of the deep reinforcement learning model in real time to expand the knowledge base.

According to another aspect of the present invention, there is provided a rehabilitation training prescription adaptive recommendation system based on deep reinforcement learning, including:

the human-computer interaction module is used for receiving the basic information and the case information of the patient and managing a pre-stored rehabilitation training knowledge base;

the near-infrared brain blood oxygen information acquisition module is used for acquiring near-infrared brain blood oxygen signals of a corresponding brain area of a patient and transmitting the acquired near-infrared brain blood oxygen signals to the evaluation analysis module;

the motion and physiological data acquisition module is used for acquiring motion signals and surface electromyographic signals of the limb of the patient in the motion process and transmitting the signals to the evaluation analysis module;

the evaluation analysis module is used for calculating to obtain brain function evaluation indexes and motion function evaluation indexes of different brain areas of the patient according to the near-infrared cerebral blood oxygen signals transmitted from the near-infrared cerebral blood oxygen acquisition module and the motion signals and surface electromyogram signals transmitted from the motion and physiological data acquisition module; and

and the intelligent learning and prescription recommending module is used for intelligently learning according to the patient medical record information in the human-computer interaction module and the evaluation indexes of the brain function and the motor function of the patient obtained by the evaluation and analysis module so as to output the rehabilitation training prescription and feeding back the rehabilitation training prescription to the doctor and the patient through the human-computer interaction module.

According to one embodiment, the intelligent learning and prescription recommendation module may comprise a pre-established rehabilitation training prescription knowledge base and a deep reinforcement learning model, wherein the rehabilitation training prescription knowledge base allows for modification and content addition through a knowledge base management module included in the human-computer interaction module.

According to another embodiment, the evaluation and analysis module can reflect the active participation degree of the brain and the muscle according to the activation degree of different brain areas and the amplitude information of different muscle electromyographic signals, and reflect the fatigue degree of the muscle of the patient by calculating frequency domain information such as average power frequency, median frequency and the like of the electromyographic signals.

The invention also provides a rehabilitation training prescription self-adaptive recommendation method based on deep reinforcement learning, which comprises the following steps:

1) basic information such as sex, age and medical history of a patient and medical record information such as etiology, brain injury condition, disease onset time, initial function level and disease course are input, and the medical record information comprises original image, medical examination data such as assay and initial evaluation data.

2) The near-infrared cerebral blood oxygen monitoring equipment is used for acquiring cerebral cortex blood oxygen data of different cerebral areas such as a movement area, a forehead leaf and the like in the movement training process of a patient, wherein the cerebral cortex blood oxygen data comprises local oxyhemoglobin concentration, deoxyhemoglobin concentration, blood oxygen saturation and the like. The inertia sensor, the surface electromyography sensor and the like are utilized to obtain the motion data of the affected limb of the patient, such as the acceleration, the angular velocity and the like, and the surface electromyography data of the relevant muscle.

3) In the exercise rehabilitation training process of the patient, brain function evaluation indexes such as the activation degrees, the activation modes, the functional connection between the brain areas, the laterality and the like of different brain areas are obtained by utilizing the brain blood oxygen data calculation, and the brain function of the patient is dynamically evaluated; calculating motion function evaluation indexes such as joint activity degree, motion smoothness and motion trajectory deviation by using motion data such as acceleration and angular velocity, and dynamically evaluating the motion function of the patient; obtaining muscle function indexes such as muscle strength, muscle tension and the like by utilizing the surface electromyography data; the active participation degree of the brain and the muscle is reflected by the activation degree of different brain areas and the amplitude information of different muscle electromyographic signals, and the fatigue degree of the muscle of a patient is reflected by frequency domain information such as the average power frequency, the median frequency and the like of the electromyographic signals.

4) A deep reinforcement learning model comprising a rehabilitation training prescription knowledge base based on a large amount of patient medical record information, function evaluation indexes and a doctor prescription is established in advance. Inputting the evaluation indexes of the brain function, the motor function and the muscle function obtained in the step 3) into a pre-established deep reinforcement learning model, and automatically generating a rehabilitation training prescription, which comprises a training task, a training scheme, a motor training mode, a training frequency, a training intensity and the like.

Specifically, the deep reinforcement learning model performs reinforcement learning by using the brain function and motor function evaluation indexes of the patient as states, using a rehabilitation training prescription as an action, and using a function improvement condition after training by using a current prescription as a reward. The training scheme in the rehabilitation training prescription comprises unilateral exercise training, limb linkage exercise training, exercise training + functional electrical stimulation, exercise training + transcranial magnetic stimulation, exercise training + virtual reality feedback and the like, the exercise training mode comprises active, passive, power-assisted, resistance and the like, the training frequency comprises the training times of each week and the training times of a single task in each training process, and the training intensity comprises the training time length of each training, the training task difficulty, the position, the intensity, the frequency and the like of the magnetoelectric stimulation.

Further, the deep reinforcement learning model adjusts the training prescription according to the active participation and fatigue degree of the patient obtained in the step 3).

5) Feeding back the rehabilitation training prescription generated in the step 4) to the doctor and the patient for rehabilitation training, and repeating the step 2).

Further, the evaluation index generated in the step 3) and the rehabilitation training prescription generated in the step 4) are added to the knowledge base of the deep reinforcement learning model in real time, and the knowledge base is continuously expanded.

The invention also provides a rehabilitation training prescription self-adaptive recommendation system based on deep reinforcement learning, which comprises:

and the human-computer interaction module comprises a medical record information input module, a rehabilitation training knowledge base management module and a recommended prescription display module.

Further, the medical record information input module is used for inputting basic information such as sex, age and medical history of the patient and medical record information such as etiology, brain injury condition, disease onset time, initial function level and course, and comprises original image, medical examination data such as assay and initial evaluation data.

The near-infrared brain blood oxygen information acquisition module comprises a near-infrared light source, a probe, a fixing device, an optical fiber, a data acquisition system and the like, is used for acquiring brain blood oxygen signals such as local oxyhemoglobin concentration, deoxyhemoglobin concentration, blood oxygen saturation and the like of a corresponding brain area of a patient, and transmits the acquired brain blood oxygen signals to the evaluation analysis module.

The motion and physiological data acquisition module comprises inertial sensors, myoelectric sensors and a data acquisition circuit which are distributed on different parts of the limb, and is used for acquiring acceleration, angular velocity and surface myoelectric signals of motion-related muscles of the limb of a patient in the motion process and transmitting the signals to the evaluation analysis module.

And the evaluation analysis module is used for calculating and obtaining brain function evaluation indexes such as activation degrees, activation modes, functional connections and laterality among the brain areas, joint activity degrees, motion smoothness, track deviation degrees and other motion function evaluation indexes and muscle function indexes such as muscle strength and muscle tension of the patient according to the brain blood oxygen signals such as the local oxyhemoglobin concentration, the deoxyhemoglobin concentration and the blood oxygen saturation transmitted by the near-infrared brain blood oxygen acquisition module and the acceleration, the angular velocity and the surface myoelectric signals transmitted by the motion and physiological data acquisition module. The active participation degree of the brain and the muscle is reflected by the activation degree of different brain areas and the amplitude information of different muscle electromyographic signals, and the fatigue degree of the muscle of a patient is reflected by calculating frequency domain information such as the average power frequency, the median frequency and the like of the electromyographic signals.

The intelligent learning and prescription recommending module comprises a rehabilitation training prescription knowledge base and a deep reinforcement learning model which are pre-established by utilizing information such as a large amount of patient medical record information, function evaluation indexes, training prescriptions issued by doctors and the like. The medical record information input module is used for inputting patient medical record information and the evaluation indexes of the brain function and the motor function of the patient, which are obtained by the evaluation and analysis module, so that the patient can learn intelligently, a rehabilitation training prescription based on the state of illness and the current functional state of the patient is output, and the rehabilitation training prescription is fed back to a doctor and the patient through the recommended prescription display module of the human-computer interaction module.

Specifically, the rehabilitation training prescription knowledge base in the intelligent learning and prescription recommending module allows a user with authority to modify the rehabilitation training prescription knowledge base through the knowledge base management module of the human-computer interaction module, so that the content is increased.

The invention has the beneficial effects that: by utilizing the method and the system, the training prescription can be adjusted in a real-time and self-adaptive manner according to the state of illness, motion function, training state and the like of the patient in the rehabilitation training process, the workload of a doctor for manually evaluating and adjusting the prescription is reduced, the adjustment is more dynamic and accurate compared with the prescription adjustment in a fixed period, and the efficiency of rehabilitation training is improved.

The foregoing summary is provided for the purpose of description only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present invention will be readily apparent by reference to the drawings and following detailed description.

Drawings

In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are therefore not to be considered limiting of its scope.

Fig. 1 is a general configuration diagram of a rehabilitation training prescription adaptive recommendation system based on deep reinforcement learning according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a rehabilitation training prescription adaptive recommendation system based on deep reinforcement learning according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating an application of a rehabilitation training prescription adaptive recommendation method based on deep reinforcement learning according to an embodiment of the present invention;

FIG. 4 is a flowchart illustrating deep reinforcement learning model calculation according to an embodiment of the present invention.

Detailed Description

In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.

As shown in fig. 1 and fig. 2, the rehabilitation training prescription adaptive recommendation system based on deep reinforcement learning generally comprises a human-computer interaction module 1, a near-infrared brain blood oxygeninformation acquisition module 2, a motion and physiologicaldata acquisition module 3, an evaluation analysis module 4 and an intelligent learning and prescription recommendation module 5.

The human-computer interaction module 1 is used for receiving basic information and case information of a patient and managing a pre-stored rehabilitation training knowledge base, and comprises a medical recordinformation entry module 11, a rehabilitation training knowledgebase management module 12 and a recommendedprescription display module 13.

The medical recordinformation input module 11 is used for inputting basic information such as sex, age, medical history and the like of a patient and medical record information such as etiology, brain injury condition, disease onset time, initial function level, disease course and the like, and comprises original medical examination data such as images and tests and initial evaluation data.

The near-infrared cerebral blood oxygeninformation acquisition module 2 is used for acquiring near-infrared cerebral blood oxygen signals of a corresponding cerebral region of a patient, transmitting the acquired near-infrared cerebral blood oxygen signals to the evaluation analysis module 4, and mainly comprises a near-infraredlight source 21, aprobe 22, a fixing device (head cap) 23, anoptical fiber 24 and adata acquisition system 25.

The near infraredlight source 21 and theprobe 22 are arranged at a fixed distance and fixed at corresponding positions of different brain regions of the head of the patient by a fixing device (in the figure, a head cap) 23. The near infrared light signals collected by theprobe 22 are transmitted to thedata collecting system 25 through theoptical fiber 24, the cerebral blood oxygen signals such as the local oxyhemoglobin concentration, the deoxyhemoglobin concentration, the blood oxygen saturation and the like of the corresponding cerebral area of the patient are obtained through calculation according to the intensity of the light signals at different probe positions, and the collected cerebral blood oxygen information is transmitted to the evaluation and analysis module 4.

The movement and physiologicaldata acquisition module 3 is used for acquiring movement signals and surface electromyogram signals of a patient in the limb movement process, transmitting the signals to the evaluation and analysis module 4, and comprises aninertial sensor 31, anelectromyogram sensor 32 and adata acquisition circuit 33 which are distributed at different parts of the limb.

Theinertial sensor 31 is used for acquiring information such as acceleration, angular velocity and the like of the patient in the limb movement process, theelectromyographic sensor 32 is used for acquiring surface electromyographic signals of muscles related to movement of the patient in the limb movement process, and the information is synchronously acquired and transmitted to the evaluation analysis module 4 through thedata acquisition circuit 33.

The evaluation analysis module 4 is configured to calculate, according to the cerebral blood oxygen signals, such as the local oxygenated hemoglobin concentration, the deoxygenated hemoglobin concentration, and the blood oxygen saturation, transmitted by the near-infrared cerebral bloodoxygen collection module 2, and the acceleration, the angular velocity, and the surface myoelectric signal transmitted by the motion and physiologicaldata collection module 3, brain function evaluation indexes, such as the activation degree, the activation pattern, the functional connection between the brain regions, and the laterality of the patient, the motion function evaluation indexes, such as the joint activity degree, the motion smoothness, and the trajectory deviation, and the muscle function indexes, such as the muscle strength and the muscle tension, of the patient in different brain regions. The active participation degree of the brain and the muscle is reflected by the activation degree of different brain areas and the amplitude information of different muscle electromyographic signals, and the fatigue degree of the muscle of a patient is reflected by calculating frequency domain information such as the average power frequency, the median frequency and the like of the electromyographic signals.

The intelligent learning and prescription recommending module 5 comprises a rehabilitation training prescription knowledge base and a deep reinforcement learning model which are pre-established by utilizing information such as a large amount of patient medical record information, function evaluation indexes, training prescriptions issued by doctors and the like. The system is used for intelligently learning according to the patient medical record information input by the medical recordinformation input module 11 and the evaluation indexes of the brain function, the motor function and the muscle function of the patient obtained by the evaluation analysis module 4, outputting a rehabilitation training prescription based on the state of illness and the current functional state of the patient, and feeding back the rehabilitation training prescription to a doctor and the patient through the recommendedprescription display module 13 of the human-computer interaction module, wherein the rehabilitation training prescription knowledge base in the intelligent learning and prescription recommendation module 5 allows a user with authority to modify through the knowledgebase management module 12 of the human-computer interaction module, and content is added.

As shown in fig. 3, the rehabilitation training prescription self-adaptive recommendation method based on deep reinforcement learning of the present invention includes the following steps:

and S1, entering basic information and medical record information of the patient.

The basic information includes sex, age, medical history, etc., and the medical record information includes etiology, brain damage condition, disease onset time, initial function level, disease course, and original image, test, etc. medical examination data and initial evaluation data, etc.

And S2, acquiring near infrared cerebral blood oxygen, motion and physiological information.

The near-infrared brain blood oxygen acquisition module is used for acquiring cerebral cortex blood oxygen parameters of different brain areas such as a movement area, a forehead leaf and the like in the movement training process of a patient, wherein the cerebral cortex blood oxygen parameters comprise local oxyhemoglobin concentration, deoxyhemoglobin concentration, blood oxygen saturation and the like. The inertia sensor, the surface electromyography sensor and the like are utilized to obtain the motion data of the affected limb of the patient, such as the acceleration, the angular velocity and the like, and the surface electromyography data of the relevant muscle.

And S3, evaluating the brain function and the motor function of the patient.

In the exercise rehabilitation training process of the patient, brain function evaluation indexes such as the activation degrees, the activation modes, the functional connection between the brain areas, the laterality and the like of different brain areas are obtained by utilizing the brain blood oxygen data calculation, and the brain function of the patient is dynamically evaluated.

Specifically, the method comprises the following steps: performing continuous complex wavelet transform on the near-infrared cerebral blood oxygen signals acquired by each acquisition channel, and representing the brain activation degree by using a wavelet amplitude; obtaining a frequency domain wavelet phase matrix through calculation, and performing wavelet phase coherence calculation on each two-channel near-infrared cerebral blood oxygen signal to obtain cerebral function connection indexes including cerebral function connection strength and effect connection strength; and dividing the difference of the brain function indexes of a certain hemisphere and the contralateral hemisphere by the sum of the brain function indexes of the certain hemisphere and the contralateral hemisphere to calculate the lateral deviation coefficient.

And calculating motion functions and muscle function evaluation indexes such as joint activity, motion coordination, muscle strength and muscle tension by using the motion data such as acceleration and angular velocity and the surface myoelectric data, and dynamically evaluating the motion functions and muscle functions of the patient.

Specifically, the method comprises the following steps: and calculating the muscle force and the muscle tension of corresponding muscles by using the amplitude of the electromyographic signals according to the approximate linear relation between the electromyography and the muscle force. By establishing a human body dynamics model, calculating the joint angle and the motion trail by using the acceleration and the angular velocity data of different segments of limbs, taking the maximum joint angle in motion as the joint activity degree, and reflecting the motion coordination by using the motion trail, the method comprises the following steps: deviation degree of the motion track from the target track, smoothness of the motion track and the like.

The active participation degree of the brain and the muscle is reflected by the activation degree of different brain areas and the amplitude information of different muscle electromyographic signals, and the fatigue degree of the muscle of a patient is reflected by frequency domain information such as the average power frequency, the median frequency and the like of the electromyographic signals.

And S4, recommending the prescription by using a deep reinforcement learning model based on the medical record information and the function evaluation indexes.

A rehabilitation training prescription knowledge base containing a large amount of patient medical record information, function evaluation indexes and a training prescription mapping relation prescribed by a doctor is established in advance, and the mapping relation of the medical records, the function indexes and the training prescription is used as prior knowledge. A Deep reinforcement Learning model is constructed by adopting a DQN (Deep Q-Learning) algorithm, a rehabilitation training prescription is used as an action according to the basic information and medical record information of a patient and evaluation indexes of brain function, motor function and muscle function as states, the function improvement condition after training is carried out by adopting the current prescription is used as a reward, the Learning model is trained, and the priori knowledge in a rehabilitation training prescription knowledge base is introduced in the training process to accelerate the training of the model. The brain function evaluation indexes comprise the activation degrees, the activation modes, the functional connection among brain areas, the laterality and the like of different brain areas. The motor function evaluation indexes include joint mobility, motor coordination, muscle strength, muscle tension and the like.

Inputting the brain function evaluation indexes of activation degree, activation mode, functional connection and laterality between brain areas and the like, the motion function evaluation indexes of joint activity degree, motion smoothness, trajectory deviation degree and the like and the muscle function evaluation indexes of muscle strength, muscle tension and the like of various different brain areas obtained in the step S3 into a trained deep reinforcement learning model, calculating and outputting Q values of different types or grades contained in rehabilitation training prescription items of a training task, a training scheme, a motion training mode, training frequency, training intensity and the like through the model, combining the prescription items with the highest Q value, and automatically generating a rehabilitation training prescription. Wherein: the training scheme comprises different categories of unilateral exercise training, limb linkage exercise training, exercise training + functional electrical stimulation, exercise training + transcranial magnetic stimulation, exercise training + virtual reality feedback and the like; the exercise training mode comprises different categories of active, passive, power-assisted, resistance and the like; the training frequency comprises the training times per week, the training times of a single task in each training process and the like, the training times per week comprise different grades of 1-7 times and the like, and the training times of a single task in each training process comprise different grades of 1-5 times and the like; the training intensity comprises the training time, the training task difficulty, the magnetoelectric stimulation position, the strength, the frequency and the like, the training time comprises different grades such as 10 minutes, 20 minutes, 30 minutes, 40 minutes, 50 minutes, 60 minutes and the like, the task difficulty comprises different grades such as simple, medium, difficult and difficult, the magnetoelectric stimulation position comprises different classes, and the magnetoelectric stimulation strength and the frequency comprise different grades.

In the using process, the evaluation index generated in the step S3 and the rehabilitation training prescription generated in the step S4 are added to the knowledge base of the deep reinforcement learning model in real time, the knowledge base is continuously expanded, and the model is automatically corrected according to the comparison condition of the function evaluation index before and after the training by adopting the recommended prescription.

The specific calculation method of the deep reinforcement training model is shown in fig. 4: firstly, initializing an experience pool and network weight, inputting state parameters such as medical records, function evaluation indexes and the like, and selecting an action (a training prescription) according to an epsilon-greedy strategy if the current state cannot be matched with the characteristic state in a priori knowledge base; and if the current-stage state can be matched with the characteristic state in the prior knowledge base, comprehensively judging the output action (training prescription) according to the prior action Q value and the estimated action Q value. Obtaining a reward value and the state of the next step according to the function improvement condition of the patient after the action (the training prescription) is adopted, storing the information into an experience pool, repeating the steps to train the network, and automatically updating the network weight after each training to correct the network.

On the other hand, the prior knowledge base of the rehabilitation training prescription contains the state characteristics of the active participation degree, the fatigue degree and the like of the patient, the deep reinforcement learning model adjusts the training prescription according to the active participation degree and the fatigue degree of the patient obtained in the step S3, and the training mode is changed or the training intensity is reduced under the condition that the active participation degree is reduced or the fatigue is generated after the patient is trained for a period of time by using the current training prescription is detected.

And S5, rehabilitation training prescription feedback.

Specifically, the method comprises the following steps: and (5) feeding back the rehabilitation training prescription generated in the step (S4) to the doctor and the patient for rehabilitation training, and repeating the step (S2).

Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (7)

Translated fromChinese

1.一种基于深度强化学习的康复训练处方自适应推荐方法，其中，该方法包括以下步骤：1. An adaptive recommendation method for rehabilitation training prescriptions based on deep reinforcement learning, wherein the method comprises the following steps:1)收集患者基本信息以及病历信息；1) Collect basic patient information and medical record information;2)获取患者不同脑区的脑皮层血氧数据，并且获得患者患肢的运动数据和肌电数据；2) Obtain the blood oxygen data of the cerebral cortex in different brain regions of the patient, and obtain the motion data and EMG data of the patient's affected limb;3)利用脑血氧数据计算得到患者运动康复训练过程中的包括不同脑区的激活程度、激活模式、脑区之间的功能连接、侧偏性的脑功能评价指标，以动态评估患者的脑功能，并且利用加速度、角速度的运动数据计算得到包括关节活动度、运动平滑度、运动轨迹偏离度的运动功能评价指标，以动态评估患者的运动功能，利用表面肌电数据得到包括肌力、肌张力的肌肉功能指标；3) Using the cerebral blood oxygen data to calculate the brain function evaluation indicators including the activation degree, activation mode, functional connection between brain regions, and laterality of different brain regions during the patient's exercise rehabilitation training process to dynamically evaluate the patient's brain function. function, and use the motion data of acceleration and angular velocity to calculate the motor function evaluation indicators including joint range of motion, motion smoothness, and motion trajectory deviation to dynamically evaluate the patient's motor function. Muscle function indicators of tension;4)将步骤3)得到的脑功能、运动功能和肌肉功能评价指标输入到预先建立的深度强化学习模型，以训练深度强化学习模型并自动生成康复训练处方；4) Input the evaluation indexes of brain function, motor function and muscle function obtained in step 3) into the pre-established deep reinforcement learning model to train the deep reinforcement learning model and automatically generate a rehabilitation training prescription;5)将步骤4)生成的康复训练处方反馈给医生和患者进行康复训练，5) Feeding back the rehabilitation training prescription generated in step 4) to doctors and patients for rehabilitation training,其中，步骤4)中的深度强化学习模型的训练包括根据患者的基本信息和病历信息，以脑功能、运动功能和肌肉功能评价指标作为状态，以康复训练处方作为动作，以采用当前的康复训练处方进行训练后的功能改善情况作为奖赏，来训练深度强化学习模型，并在训练过程中引入康复训练处方知识库中的先验知识，加速学习模型的训练，其中康复训练处方包括训练任务、训练方案、运动训练模式、训练频率、训练强度，并且其中预先建立包含大量患者病历信息、功能评估指标与医生开具的训练处方映射关系的康复训练处方知识库，以病历、功能指标和训练处方的映射关系作为先验知识，Wherein, the training of the deep reinforcement learning model in step 4) includes taking the evaluation indexes of brain function, motor function and muscle function as the state, taking the rehabilitation training prescription as the action, and using the current rehabilitation training according to the basic information and medical record information of the patient. The functional improvement after the prescription is trained is used as a reward to train the deep reinforcement learning model, and the prior knowledge in the rehabilitation training prescription knowledge base is introduced in the training process to accelerate the training of the learning model. The rehabilitation training prescription includes training tasks, training Scheme, exercise training mode, training frequency, training intensity, and a rehabilitation training prescription knowledge base that contains a large amount of patient medical record information, functional evaluation indicators and the mapping relationship between the training prescription issued by the doctor is pre-established, with the mapping of medical records, functional indicators and training prescriptions. relation as prior knowledge,步骤4)中的深度强化学习模型的训练包括首先初始化经验池和网络权重，输入病历和各功能评价指标的状态参数，并且如果当前状态不能匹配到先验知识库中的特征状态，则根据ε-greedy策略选取训练处方；如果当期状态能够匹配到先验知识库中的特征状态，则根据先验动作Q值和预估动作Q值综合判断输出训练处方，根据采用该训练处方后患者的功能改善情况得到奖赏值和下一步的状态，并将这些信息存入经验池，重复以上步骤训练网络，每次训练后自动更新网络权重以修正网络。The training of the deep reinforcement learning model in step 4) includes first initializing the experience pool and network weights, inputting medical records and state parameters of each functional evaluation index, and if the current state cannot match the feature state in the prior knowledge base, according to ε - The greedy strategy selects the training prescription; if the current state can match the characteristic state in the prior knowledge base, the training prescription is output according to the comprehensive judgment of the prior action Q value and the estimated action Q value, and the function of the patient after the training prescription is adopted. Improve the situation to get the reward value and the state of the next step, and store this information into the experience pool, repeat the above steps to train the network, and automatically update the network weight after each training to correct the network.2.根据权利要求1所述的基于深度强化学习的康复训练处方自适应推荐方法，其中，将步骤3)得到的脑功能评价指标、运动功能评价指标以及肌肉功能评价指标输入到训练好的深度强化学习模型，经模型计算输出各康复训练处方中包含的不同类别或等级的Q值，将Q值最高的处方项组合，自动生成康复训练处方，其中Q值为对应动作优劣的数值化表示。2. The self-adaptive recommendation method for rehabilitation training prescriptions based on deep reinforcement learning according to claim 1, wherein the brain function evaluation index, motor function evaluation index and muscle function evaluation index obtained in step 3) are input into the trained depth. Reinforcement learning model, the model calculates and outputs the Q values of different categories or levels contained in each rehabilitation training prescription, and combines the prescription items with the highest Q value to automatically generate a rehabilitation training prescription, where the Q value is a numerical representation of the pros and cons of the corresponding actions .3.根据权利要求1所述的基于深度强化学习的康复训练处方自适应推荐方法，其中，利用步骤2)得到的脑血氧参数数据和肌电数据进一步分析得到患者训练过程中的大脑和肌肉的主动参与度和疲劳程度，并且步骤4)中的深度强化学习模型能够根据患者的主动参与度和疲劳程度来来调整训练处方。3. The rehabilitation training prescription adaptive recommendation method based on deep reinforcement learning according to claim 1, wherein, using step 2) obtained cerebral blood oxygen parameter data and myoelectric data to further analyze to obtain the brain and muscles in the patient training process. active participation and fatigue level, and the deep reinforcement learning model in step 4) can adjust the training prescription according to the active participation and fatigue level of the patient.4.根据权利要求3所述的基于深度强化学习的康复训练处方自适应推荐方法，其中，以不同脑区的激活程度和不同肌肉肌电信号的幅值信息反映大脑和肌肉的主动参与度，以肌电信号的频域信息反映患者肌肉的疲劳程度。4. The self-adaptive recommendation method for rehabilitation training prescriptions based on deep reinforcement learning according to claim 3, wherein the active participation of the brain and the muscles is reflected with the activation degrees of different brain regions and the amplitude information of different muscle EMG signals, The degree of muscle fatigue of the patient is reflected by the frequency domain information of the EMG signal.5.一种基于深度强化学习的康复训练处方自适应推荐系统，包括：5. An adaptive recommendation system for rehabilitation training prescriptions based on deep reinforcement learning, comprising:人机交互模块，用于接收患者的基本信息和病例信息并且管理预先存储的康复训练知识库；The human-computer interaction module is used to receive the patient's basic information and case information and manage the pre-stored rehabilitation training knowledge base;近红外脑血氧信息采集模块，用于采集患者相应脑区的近红外脑血氧信号，并将采集到的近红外脑血氧信号传输至评估分析模块；The near-infrared cerebral blood oxygen information acquisition module is used to collect the near-infrared cerebral blood oxygen signal of the corresponding brain region of the patient, and transmit the collected near-infrared cerebral blood oxygen signal to the evaluation and analysis module;运动及生理数据采集模块，用于采集患者肢体运动过程中的运动信号和表面肌电信号，并将这些信号传输至评估分析模块；The movement and physiological data acquisition module is used to collect movement signals and surface EMG signals during the movement of the patient's limbs, and transmit these signals to the evaluation and analysis module;评估分析模块，用于根据从近红外脑血氧采集模块传输的近红外脑血氧信号以及从运动及生理数据采集模块传输的运动信号和表面肌电信号，计算得到患者不同脑区的脑功能、运动功能和肌肉功能评价指标；以及The evaluation and analysis module is used to calculate the brain function of different brain regions of the patient according to the near-infrared cerebral blood oxygen signal transmitted from the near-infrared cerebral blood oxygen acquisition module and the motion signal and surface EMG signal transmitted from the exercise and physiological data acquisition module , motor function and muscle function assessment indicators; and智能学习与处方推荐模块，用于根据人机交互模块中的患者病历信息以及评估分析模块得到的患者脑功能、运动功能和肌肉功能评价指标进行智能学习，以输出康复训练处方，并通过人机交互模块反馈给医生和患者，The intelligent learning and prescription recommendation module is used for intelligent learning according to the patient's medical record information in the human-computer interaction module and the evaluation indicators of the patient's brain function, motor function and muscle function obtained by the evaluation and analysis module, so as to output the rehabilitation training prescription and pass the human-computer Interactive module feedback to doctors and patients,其中，利用脑血氧数据计算得到包括不同脑区的激活程度、激活模式、脑区之间的功能连接、侧偏性的脑功能评价指标，以动态评估患者的脑功能，并且利用加速度、角速度的运动数据计算得到包括关节活动度、运动平滑度、运动轨迹偏离度的运动功能评价指标，以动态评估患者的运动功能，利用表面肌电数据得到包括肌力、肌张力的肌肉功能指标；Among them, the brain function evaluation indexes including the activation degree, activation mode, functional connection between brain regions, and laterality of different brain regions are calculated by using cerebral blood oxygen data to dynamically evaluate the brain function of patients. The motor function evaluation indexes including joint range of motion, motion smoothness, and motion trajectory deviation are calculated from the motion data, so as to dynamically evaluate the patient's motor function, and the surface electromyography data are used to obtain the muscle function indexes including muscle strength and muscle tension;智能学习与处方推荐模块包含预先建立的康复训练处方知识库和深度强化学习模型，深度强化学习模型的训练包括首先初始化经验池和网络权重，输入病历和各功能评价指标的状态参数，并且如果当前状态不能匹配到先验知识库中的特征状态，则根据ε-greedy策略选取训练处方；如果当期状态能够匹配到先验知识库中的特征状态，则根据先验动作Q值和预估动作Q值综合判断输出训练处方，根据采用该训练处方后患者的功能改善情况得到奖赏值和下一步的状态，并将这些信息存入经验池，重复以上步骤训练网络，每次训练后自动更新网络权重以修正网络。The intelligent learning and prescription recommendation module includes a pre-established rehabilitation training prescription knowledge base and a deep reinforcement learning model. The training of the deep reinforcement learning model includes first initializing the experience pool and network weights, inputting the medical records and the state parameters of each function evaluation index, and if the current If the state cannot match the feature state in the prior knowledge base, the training prescription is selected according to the ε-greedy strategy; if the current state can match the feature state in the prior knowledge base, then the prior action Q value and the estimated action Q are selected according to the current state. The value comprehensively judges and outputs the training prescription, and obtains the reward value and the next state according to the functional improvement of the patient after using the training prescription, and stores the information in the experience pool, repeats the above steps to train the network, and automatically updates the network weight after each training. to fix the network.6.根据权利要求5所述的基于深度强化学习的康复训练处方自适应推荐系统，其中，智能学习与处方推荐模块包含预先建立的康复训练处方知识库和深度强化学习模型，其中康复训练处方知识库允许通过人机交互模块中所包括的知识库管理模块进行修改、增加内容。6. The deep reinforcement learning-based adaptive recommendation system for rehabilitation training prescriptions according to claim 5, wherein the intelligent learning and prescription recommendation module comprises a pre-established rehabilitation training prescription knowledge base and a deep reinforcement learning model, wherein the rehabilitation training prescription knowledge The library allows modification and addition of content through the knowledge base management module included in the human-computer interaction module.7.根据权利要求5所述的基于深度强化学习的康复训练处方自适应推荐系统，其中，评估分析模块以不同脑区的激活程度和不同肌肉肌电信号的幅值信息反映大脑和肌肉的主动参与度，通过计算肌电信号的频域信息反映患者肌肉的疲劳程度。7. The adaptive recommendation system for rehabilitation training prescriptions based on deep reinforcement learning according to claim 5, wherein the evaluation and analysis module reflects the active activities of the brain and muscles with the activation degrees of different brain regions and the amplitude information of different muscle EMG signals. The degree of participation reflects the fatigue degree of the patient's muscles by calculating the frequency domain information of the EMG signal.

Images