Detailed Description
The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The intelligent emergency decision system based on multi-mode fusion is shown in fig. 1, and is characterized by comprising a main control unit 10, wherein the main control unit 10 is connected with a display unit 20, a connection unit 30, an interaction unit 40 and a cross-linking unit 50, the main control unit 10 stores basic information and detailed information of a patient, the connection unit 30 acquires patient life monitoring data through a medical instrument, the display unit 20 is used for displaying the basic information of the patient and the patient life monitoring data, the interaction unit 40 is used for receiving and recording the current condition of the patient dictated by a doctor, the cross-linking unit 50 is used for connecting with the Internet of things of the hospital to acquire medicine information, when the main control unit 10 judges whether the patient has health hidden dangers through the patient life monitoring data, if the health hidden dangers are judged to have the health hidden dangers, the main control unit 10 predicts possible symptoms according to the patient life monitoring data and generates predicted emergency contents according to all predicted symptoms, when the doctor judges that the patient has the illness and needs emergency, the doctor calls the predicted emergency contents for doctor, if the predicted emergency contents are predicted to have the illness symptoms of the patient, the doctor calls the predicted emergency contents of the corresponding symptoms, if the patient's emergency contents are not the patient's illness, the doctor inputs the current emergency conditions and symptoms of the patient, the patient's health status and symptoms through the interaction unit 40, the patient interface unit, the medical condition, the medical information, the doctor, the medical information, and the medical information.
The main control unit 10 judges whether the patient has health hidden trouble or not by collecting life monitoring data of the patient, so that possible symptoms of the patient are predicted, and predicted emergency content is generated in advance, when the patient needs emergency, a doctor can observe the symptoms of the patient, check whether the main control unit 10 has the predicted emergency content which is generated in advance for reference, if so, the main control unit 10 can refer to the predicted emergency content for emergency, and the main control unit 10 records the emergency process. If not, the doctor inputs the on-line generation scheme of the symptoms, or directly carries out emergency treatment according to the specific symptoms of the patient, and the main control unit 10 also records the emergency treatment process. The time for generating the content by the main control module can be saved when the emergency occurs by generating the predicted emergency content in advance, and in addition, the main control module records the emergency content, so that the legal risk prevention and control effect is achieved, the time axis precision is high, and the digital signature and CA authentication meet the judicial authentication requirement. After each emergency data is recorded, the emergency data can be used as a reference in the next prediction, so that the prediction accuracy is improved.
As shown in fig. 2 and 3, the system is installed on an integrated computer (touch screen) +an intelligent rescue vehicle. When in use, the detailed information such as medical history, past history, main diagnosis and treatment pass and auxiliary examination can be rapidly extracted by pressing the red start key of the ambulance computer and selecting the emergency patient by the touch screen of the display unit 20. The system can quickly integrate information such as the current condition of a patient and life monitoring data by voice or text, generate a preliminary first-aid scheme in 10-20 seconds according to evidence-based medicine through a disease analysis large model, open a special first-aid channel of a power calculation server, and ensure the operation speed by system priority.
Specifically, the main control unit 10 synchronizes 12-lead waveform data such as ECG, spO2, invasive blood pressure and the like of the monitor in real time through the connection unit 30, the sampling rate is more than or equal to 500Hz, and the connection mode is realized through Bluetooth or WiFi.
Specifically, the interaction unit 40 is a medical noise reduction microphone array, supports 10 meters far-field pickup, integrates a medical special voice recognition engine, and can recognize the technical terms of 'amiodarone 150mg static pushing'.
Specifically, the crosslinking unit 50 may read the data of medicine inventory, incompatibility, etc. of the intelligent medicine cabinet in real time.
Specifically, a multi-mode fusion algorithm framework of the main control module adopts a model of layering feature fusion and dynamic weight distribution. The method comprises a bottom layer characteristic layer, wherein the life data extracts time sequence characteristics through an LSTM network, a voice instruction is analyzed into medical behavior intention vectors through a BERT model, and medicine information is embedded into the medical behavior intention vectors through a knowledge graph to generate structural characteristics. The middle layer decision layer builds a cross-modal attention mechanism based on a transducer architecture, and introduces clinical time window constraint (such as 90 minutes gold treatment period after symptoms of myocardial infarction patients appear) to dynamically adjust the weight of each mode.
The master control module comprises a multi-mode fusion algorithm, and the multi-mode fusion process can be formalized by the following mathematical expression:
Is provided withInput data of different modesIs provided withRepresent the firstData of seed modeRepresent the firstData of a modality. Data of modalities such as patient life monitoring data, medication information prediction, doctor input of patient current condition and disorder through the interaction unit 40, etc.
Firstly, each mode data is converted into a characteristic representation through a respective characteristic extraction network:
Wherein the method comprises the steps ofIs the firstA characteristic representation of the seed modality,Is the firstA characteristic representation of the seed modality,AndA feature extraction network representing corresponding modality data.
Next, interactions between different modalities are handled using a cross-modality attention mechanism in a transducer architecture, for the modalitiesAnd modalityThe attention calculations between can be expressed as:
Wherein the method comprises the steps ofIs a query matrix for extracting query features from the current modality,Is a key matrix for storing key information that can be matched,Is the dimension of the key vector and,Is a modality specific matrix of learnable parameters obtained by learning the Q-map,Is a modality specific matrix of learnable parameters obtained by learning the K-map.
The attention output can be expressed as:
Wherein V is represented as a matrix of values, whereinIs a mode-specific matrix of learnable parameters responsible for mapping the original features to a value space, a value matrix =By a matrix of learnable parametersCharacterization of modality jObtained by performing linear transformation.
Aggregation of cross-modal attention is expressed as:
The value matrix assumes the role of the information content provider in the attention mechanism. When the similarity calculation of the query matrix Q and the key matrix K is completed, attention weight is generatedThese weights are then applied to the value matrix to obtain the actual information content that ultimately needs to be extracted. Aggregation of cross-modal attentionContains the interaction information between modes, compared with the original characteristic representationThere is more rich semantics.
Then, introducing a dynamic weight distribution mechanism of clinical time window constraint, and carrying out weighted fusion on each mode characteristic:
Wherein the method comprises the steps ofIs and clinical time windowA related scoring function is provided which is a function of the scoring,Is the firstThe dynamic weight of each mode is used for dynamically adjusting the importance of each mode according to the constraint condition of the time window under different clinical scenes. For example, during the golden 90 minute treatment period of a patient with myocardial infarction, the weight of the electrocardiographic data may be automatically increased by the system, while at other points in time, other vital sign data may be weighted higher.
The final multimodal fusion feature is expressed as:
The fusion featureWill be used as an input to a subsequent decision model to obtain emergency content.
Federal learning optimization, namely constructing a federal learning platform in a data center in a hospital, uploading encrypted model gradients (non-original data) by nodes of each department, and updating a global fusion model regularly to meet the data privacy requirement of a medical and health data management method. The real-time guarantee mechanism adopts DDS (data distribution service) technology to realize millisecond-level data synchronization, and the end-to-end delay is less than 200ms. The double buffer queues are designed, and emergency data (such as ventricular fibrillation waveforms) trigger interrupt priority scheduling, so that key information processing delay is ensured to be less than 50ms.
The main control unit 10 generates a clinical decision engine for realizing the prediction of emergency contents according to an intelligent emergency scheme, which comprises the steps of constructing a dynamic knowledge map, firstly fusing multiple databases by multi-source knowledge, comprising a basic rule base, namely integrating UpToDate clinical guidelines, an AHA cardiopulmonary resuscitation flow, an expert consensus for diagnosis and treatment of acute poisoning in China and the like, authoritative guidelines to construct a decision tree containing 8000+ medical rules, analyzing 20000+ emergency medical records in the recent 5 years through a natural language processing technology, and extracting a characteristic diagnosis and treatment path of a hospital (such as thrombolysis time window adjustment rules for myocardial infarction patients in a plateau region). Multiple databases update the evidence in real time, namely, interfacing PubMed Clinical Queries, and automatically synchronizing the latest clinical research evidence every week (the evidence grade is more than or equal to grade 2A).
And then strengthening the learning decision model, and designing a reward function of the emergency measure sequence by taking the grading change rate, the lactic acid clearance rate and the like of the patient APACHEII as state variables. And a PPO (near-end strategy optimization) algorithm is adopted to train a decision model, and the influence prediction of different emergency treatment schemes on the 28-day mortality is simulated.
The main control unit 10 generates emergency content through an emergency decision model, and the mathematical expression is as follows:
The master control unit 10 defines a state spaceIncluding various physiological indexes and clinical parameters of the patient, such as multidimensional feature vectors of APACHE II score, lactic acid level, hemodynamic parameters and the like. Action spaceRepresents a discrete set of clinical interventions including dosing regimens, surgical interventions, life support measures, and the like. Reward functionThe medical value of performing action a in state S is quantified, typically mapped to a patient prognostic indicator.
In the state spaceBased on the input characteristic Z, in the action spaceAnd updating the strategy according to R (s, a). The method comprises the steps of obtaining a multi-mode fusion Z as an input characteristic of an emergency decision model, taking S formed by physiological indexes and clinical parameters of a patient as a 'state' in reinforcement learning, selecting an action A based on the input characteristic Z in the state S, and updating a strategy according to R (S, a).
The emergency decision is modeled as a Markov Decision Process (MDP) with the goal of finding the optimal strategyMaximizing the desired jackpot:
Wherein the method comprises the steps ofIs a discount factor, adjusts the trade-off parameter of the recent income and the distant income, reflects the clinical time window constraint, and T is a sequence position identifier from 0 to the end of T, wherein each T corresponds to a specific time point in the emergency process; Is the decision sequence length; is a strategy for making decisions in different states; is the expected value under policy pi. The formula is mainly used for solving the optimal strategy. Optimal strategyRepresenting the optimal course of action that the system should take, corresponds to an "ideal first aid guideline".
Updating the policy of the emergency decision by optimizing the objective function using the PPO algorithm:
Wherein the method comprises the steps ofIs a parameter of the emergency decision model,Is the probability ratio of the new strategy to the old strategy,Is an estimate of the dominance function,Is a clipping function, which is a clipping function,Is a cutting parameter, and the cutting parameter is a cutting parameter,The experience of said time t is expected.
The visual decision flow part code is as follows:
decision pseudocode for patient with # acute chest pain
def chest_pain_decision(patient_data):
if ecg_has_st_elevation(patient_data):
Calculation of time to onset of symptom #
if time_since_onset<120min:
# Catheter Chamber State query
if can_perform_pci(patient_data):
# Initiate PCI preoperative preparation procedure
return generate_pci_protocol()
else:
Thrombolysis scheme conforming to STEMI diagnosis and treatment guide
return generate_thrombolysis_protocol()
else:
return stable_angina_management()
else:
Dynamic monitoring of # myocardial injury markers
if troponin>99th_percentile:
return nstemi_treatment()
else:
# Initiate other etiology investigation procedures
return rule_out_acs()
Further, the main control unit 10 is further connected to a dedicated telephone line unit 60, and the dedicated telephone line unit 60 establishes dedicated telephone lines with consultants, pharmacies and blood banks for one-key voice call. The telephone special line unit 60 selects an operator in China Mobile/China Unicom/China telecom, opens a special voice telephone special line for emergency, sets a one-key call consultation system-cloud connection, embeds a hospital emergency number and a call template, and supports one-key voice telephone call consultation doctors/pharmacies/blood banks. Consultation items requiring emergency calls are listed in the emergency regimen. According to different requirements, a standardized help-seeking short message and enterprise micro-signal notification are pushed simultaneously, for example, a tracheal cannula of anesthesia department is called, namely, your patient is called, an emergency tracheal cannula is needed for a 12-bed in a 222-ward patient area, and the patient is asked to go to immediately; clinical departments include your own, 222 ward 12 beds need urgent consultation, please go to the hospital immediately, blood banks, 222 ward 12 beds need urgent transfusion, blood of the patient is of blood type A, blood is cross-sent immediately, please prepare blood immediately and send as soon as possible, 100 calls, 222 ward 12 beds need urgent blood sample to go to the hospital immediately, pharmacy, you't go to the hospital, 222 ward 12 beds need 5 norepinephrine, please send immediately, etc. Thereby realizing the quick starting of the multi-disciplinary team cooperation rescue.
The medication advice may give the name of the medication currently needed by the patient and display the location information of the medication according to the crosslinking unit 50, which is convenient and quick to obtain. The dose recommendations for providing medication are automatically calculated based on body weight. The medicine warning is such as "warfarin is being used and heparin is cautiously used". The most recent storage point is shown when the medicine is lacking, for example, 10 letters are stored in an emergency pharmacy.
The integrated touch display unit 20 forms a physical binding with the mobile rescue vehicle and can be adjusted at multiple angles.
The drug dosage recommendation is calculated according to the weight of the patient, the drug warning is obtained according to the detailed information of the patient, and the nearest storage point of the drug is obtained according to the crosslinking unit 50 to obtain the position information of the corresponding drug.
The display unit 20 displays the standard emergency procedure step by step according to the emergency plan, supports touch scaling of the screen, and highlights the directions in steps on the screen.
The main control unit 10 generates an emergency record according to the emergency procedure in accordance with the time axis, records the medication time and defibrillation time as key nodes in the time axis, and records an outgoing examination, a department of a round, etc. in the time axis. Each stage of the time axis refers to the following table.
The granularity of the time axis is accurate to the operation time (millisecond level) +the operation main body (card-brushing authentication) +the equipment number (unique medical equipment code), such as 2025-04-30T 14:23:15.452+08:00|nurse A|bedside monitor #CN2024001|SpO2 rises from 88% to 94%.
Typical scenarios for judicial authentication include compliance verification of cardiopulmonary resuscitation operations by computing key indicators such as compression frequency (100-120 times/min), interruption time (10 seconds). And judging medication time sequence disputes, namely precisely tracing the time sequence of 'epinephrine first static pushing' and 'defibrillator charging completion', and solving the dispute of a rescue flow.
After the interaction unit 40 records the current condition of the patient dictated by the doctor, the main control unit 10 analyzes the voice command of the current condition of the patient, generates a standardized electronic medical advice, the nurse confirms the execution of the electronic medical advice through the touch screen of the display unit 20, and the main control unit 10 records the execution of the electronic medical advice by the nurse in a time axis.
The main control unit 10 records voiceprints of respective doctors and nurses, and the interaction unit 40 distinguishes doctors and nurses through voiceprint recognition. For example, the nurses say 09:25, epinephrine 1mg static push, 09:30,200J biphase electric defibrillation 1 time, and the like, then the interaction unit 40 converts the corresponding nurse say content into words through voice, and the main control unit 10 generates a rescue time shaft with a time stamp in real time, thereby facilitating the supplementary recording of the structured data after the rescue is finished, and the integrity of the data can be checked by combining with time logic.
The main control unit 10 generates an emergency file after the emergency is finished, is used for automatically sorting the emergency flow according to a time axis, and is combined with time logic to check the data integrity of the flow, generates an emergency report which accords with medical standards, and triggers the subsequent flow according to the emergency result. After the rescue is finished, a rescue completion button is clicked, then the main control unit 10 automatically sorts the rescue time line, and the data integrity is checked by combining time logic, so that an emergency report which accords with the medical standard is generated, and the emergency report is checked and confirmed by medical staff. And automatically generating a consultation list according to the consultation call and the consultation opinion, wherein the consultation list is consistent with the consultation list in the current electronic medical record system, and can be imported and modified by one key by a doctor, and finally, the whole-course emergency data is encrypted and archived. The follow-up rescue procedure comprises automatic filling of contents such as department of rotation, death evidence and the like. The main control unit 10 also traces back the emergency procedure according to the emergency file, and automatically marks the link to be improved, for example, "defibrillation preparation is delayed by 50 seconds".
The main control unit 10 is also connected with an evaluation unit 70, the evaluation unit 70 is preset with a first-aid item corpus c= [ C1, C2...ci ], ci represents the i-th first-aid item, each element of the first-aid item corpus C is respectively provided with an important score Si, and Si represents the important score of the i-th first-aid item;
The evaluation unit 70 forms a content aggregate P, p= [ P1, P2.., pi ] from each emergency item on the emergency record acquisition time axis, pi represents the i-th emergency content on the time axis, T, t= [ T1, T2., ti ], ti represents the time spent for the i-th emergency content on the time axis, maps each element of the content aggregate P onto the emergency item corpus C to obtain an important score Si of each element of the content aggregate P, each element of the content aggregate P obtains a weight B, b= [ B1, B2., bi ] of each element from the important score Si, bi represents the weight of the i-th emergency content of the content aggregate P, and the evaluation unit finds the evaluation score from the emergency items by:
where Score represents an evaluation Score, bi represents a weight of the ith emergency content of the content collection P, and Ti represents a time spent by the ith emergency content on a time axis.
To improve the accuracy and comparability of the evaluation, the time integration set T can be normalized to define normalized timeThe method comprises the following steps:
Wherein the method comprises the steps ofAndRespectively, a minimum value and a maximum value of time spent for all emergency projects. The normalized evaluation score is calculated as:
Wherein the method comprises the steps ofIs a small constant to avoid the case where the denominator is zero.
The emergency record of each time searches for the corresponding emergency project weight in the preset emergency project total set according to the emergency projects on the time axis, so that the weight of each emergency project in the emergency record of this time is calculated, the weight can be obtained through comprehensive calculation of the score of a certain emergency project in the emergency record accounting for all emergency projects, the score is lower as the time spent by the emergency project is longer, and elements of the time total set T can be normalized and then participate in calculation. And finally, calculating the evaluation score of a certain emergency record, wherein the evaluation score is used for evaluating the score obtained by each emergency, and the score comprises the score of each emergency item, so that the repeated operation can be carried out on each item, and an optimized process can be found out. The evaluation unit calculates an average score of the emergency records by the following process;
Wherein, theThe average score of the emergency records is represented, thereby being used for comparing the score size of each emergency record and judging the performance of medical staff in different emergency records.
To further refine the evaluation mechanism, a time sensitivity factor may be introducedWeighting the timeliness requirements of different emergency projects:
Wherein the method comprises the steps ofIs the firstThe time sensitivity factor of the emergency content is given a higher value for time sensitive procedures (e.g., defibrillation, endotracheal intubation, etc.), and a lower value for relatively less time sensitive procedures.An estimated latitude is added for estimating and quantifying the time urgency dimension of each emergency procedure, two equally important procedures, the sameThe values, possibly with completely different time sensitivity requirements, raw weighting factorsThe general clinical importance of a procedure is measured, while βi is specific to the speed at which the procedure must be performed. Timeliness scoreThe performance of medical staff in different first aid records can be reasonably evaluated.
The application has the beneficial effects that 1, the rescue efficiency is obviously improved. 2. The management accuracy is enhanced, and the accurate management and control of the whole rescue process is realized. 3. Decision science assurance-reducing procedural errors by Clinical Decision Support Systems (CDSS). 4. And the legal risk prevention and control is high in time axis precision, and the digital signature and CA authentication meet the requirement of judicial authentication. 5. Resource integration optimization, namely shortening the response time of rescue related communication and improving the multi-system data penetration rate. 6. And the quality control closed loop automatically triggers a quality control check point, so that the problem tracing efficiency is improved, the current rescue step can be optimized in real time, and high-fidelity training data can be provided for subsequent quality improvement.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the present invention may occur to one skilled in the art without departing from the principles of the present invention and are intended to be within the scope of the present invention.