技术领域Technical Field
本申请涉及智能分析领域，且更为具体地，涉及一种多发性骨髓瘤心脏淀粉样变患者心理状态分析系统及方法。The present application relates to the field of intelligent analysis, and more specifically, to a system and method for analyzing the psychological state of patients with multiple myeloma cardiac amyloidosis.
背景技术Background Art
多发性骨髓瘤是一种涉及浆细胞的血液癌症，这些异常的浆细胞会侵犯骨髓并影响正常的血细胞生成。心脏淀粉样变是多发性骨髓瘤的一种罕见并发症，由异常的免疫球蛋白轻链沉积在心脏组织中引起，导致心脏结构和功能受损。Multiple myeloma is a blood cancer involving plasma cells, which invade the bone marrow and affect normal blood cell production. Cardiac amyloidosis is a rare complication of multiple myeloma caused by abnormal immunoglobulin light chains deposited in heart tissue, resulting in damage to heart structure and function.
由于心脏淀粉样变性可能导致心力衰竭和其他严重的心脏症状，这些症状及其对生活质量的影响可能对患者的心理状态产生负面影响，包括焦虑、抑郁，因此，心理状态分析对于这类患者来说至关重要。Because cardiac amyloidosis may lead to heart failure and other serious cardiac symptoms, these symptoms and their impact on quality of life may have a negative impact on the patient's psychological state, including anxiety and depression. Therefore, psychological status analysis is crucial for such patients.
然而，传统心理状态评估方法通常依赖于问卷调查或临床访谈，这些方法可能缺乏对细微生理变化的敏感性，如心率变异性或血压的微小波动，而这些细微的变化可能与心理压力相关。此外，由于心理状态可能随时间和环境快速变化，而传统方法可能无法实时捕捉心理状态的快速变化，且无法在没有明显症状的情况下识别潜在的心理问题。However, traditional methods of assessing mental status usually rely on questionnaires or clinical interviews, which may lack sensitivity to subtle physiological changes, such as heart rate variability or small fluctuations in blood pressure, which may be associated with psychological stress. In addition, since mental status may change rapidly over time and in the environment, traditional methods may not be able to capture rapid changes in mental status in real time and cannot identify potential psychological problems in the absence of obvious symptoms.
因此，期望一种优化的多发性骨髓瘤心脏淀粉样变患者心理状态分析系统，其能够基于患者的超声心动图和生理数据的联合分析来实时捕获并分析患者的心理状态。Therefore, an optimized psychological state analysis system for multiple myeloma cardiac amyloidosis patients is desired, which can capture and analyze the psychological state of the patient in real time based on a joint analysis of the patient's echocardiographic and physiological data.
发明内容Summary of the invention
为了解决上述技术问题，提出了本申请。In order to solve the above technical problems, this application is proposed.
根据本申请的一个方面，提供了一种多发性骨髓瘤心脏淀粉样变患者心理状态分析系统，其包括：According to one aspect of the present application, a psychological state analysis system for patients with multiple myeloma cardiac amyloidosis is provided, comprising:
患者超声心动图获取模块，用于获取被监测多发性骨髓瘤心脏淀粉样变患者的超声心动图；A patient echocardiogram acquisition module, used to acquire an echocardiogram of a monitored multiple myeloma cardiac amyloidosis patient;
患者生理数据采集模块，用于获取由传感器组采集的所述被监测多发性骨髓瘤心脏淀粉样变患者的生理数据的时间序列，其中，所述生理数据包括心率值和血压值；A patient physiological data acquisition module, used for acquiring a time series of physiological data of the monitored multiple myeloma cardiac amyloidosis patient acquired by the sensor group, wherein the physiological data includes a heart rate value and a blood pressure value;
生命体征数据时序关联模块，用于将所述生理数据的时间序列进行数据规整以得到患者心率时序输入向量和患者血压时序输入向量后通过患者生命体征时序特征提取器以得到患者心率时序关联特征向量和患者血压时序关联特征向量；A vital sign data time series association module, used for regularizing the time series of the physiological data to obtain a patient heart rate time series input vector and a patient blood pressure time series input vector, and then passing the vector through a patient vital sign time series feature extractor to obtain a patient heart rate time series association feature vector and a patient blood pressure time series association feature vector;
心率-血压时序特征交互融合模块，用于将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量通过基于门控响应机制的生命体征多模态参数时序动态交互融合模块以得到多模态生命体征时序语义动态交互融合特征向量；A heart rate-blood pressure time series feature interactive fusion module, used to obtain a multimodal vital sign time series semantic dynamic interactive fusion feature vector by passing the patient's heart rate time series associated feature vector and the patient's blood pressure time series associated feature vector through a vital sign multimodal parameter time series dynamic interactive fusion module based on a gated response mechanism;
超声心动图语义特征提取模块，用于将所述超声心动图通过超声心动图语义特征提取器以得到超声心动图语义特征向量；An echocardiogram semantic feature extraction module, used for passing the echocardiogram through an echocardiogram semantic feature extractor to obtain an echocardiogram semantic feature vector;
生命体征时序-超声心动图特征交互模块，用于将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器以得到超声心动语义影响下患者心理状态时序表征向量作为超声心动语义影响下患者心理状态时序表征特征；A vital sign time series-echocardiogram feature interaction module, used for obtaining a patient's psychological state time series representation vector under the influence of echocardiogram semantics as a patient's psychological state time series representation feature under the influence of echocardiogram semantics by using the multimodal vital sign time series semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector through a vital sign-echocardiogram semantic dynamic response interaction encoder based on an adaptive response feature differentiation network;
心理状态等级评估模块，用于基于所述超声心动语义影响下患者心理状态时序表征特征，得到检测结果，所述检测结果用于表示被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级标签。The psychological state level assessment module is used to obtain a detection result based on the temporal representation characteristics of the patient's psychological state under the influence of the ultrasonic cardiography semantics, and the detection result is used to represent the psychological state level label of the monitored multiple myeloma cardiac amyloidosis patient.
在上述多发性骨髓瘤心脏淀粉样变患者心理状态分析系统中，所述生命体征数据时序关联模块，包括：In the above-mentioned psychological state analysis system for patients with cardiac amyloidosis of multiple myeloma, the vital sign data time series association module includes:
数据规整单元，用于将所述生理数据的时间序列按照时间维度和样本维度进行数据规整以得到所述患者心率时序输入向量和患者血压时序输入向量；A data regularization unit, used for regularizing the time series of the physiological data according to the time dimension and the sample dimension to obtain the patient's heart rate time series input vector and the patient's blood pressure time series input vector;
患者生命体征时序特征提取单元，用于将所述患者心率时序输入向量和所述患者血压时序输入向量通过基于1D_CNN网络的患者生命体征时序特征提取器以得到所述患者心率时序关联特征向量和所述患者血压时序关联特征向量。The patient vital sign timing feature extraction unit is used to pass the patient heart rate timing input vector and the patient blood pressure timing input vector through a patient vital sign timing feature extractor based on a 1D_CNN network to obtain the patient heart rate timing associated feature vector and the patient blood pressure timing associated feature vector.
在上述多发性骨髓瘤心脏淀粉样变患者心理状态分析系统中，所述心率-血压时序特征交互融合模块，包括：In the above-mentioned psychological state analysis system for patients with multiple myeloma cardiac amyloidosis, the heart rate-blood pressure time series feature interactive fusion module includes:
心率-血压时序关联级联单元，用于将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量进行级联处理以得到心率-血压时序关联级联特征向量；A heart rate-blood pressure time series association cascade unit, used for cascading the patient's heart rate time series association feature vector and the patient's blood pressure time series association feature vector to obtain a heart rate-blood pressure time series association cascade feature vector;
心率-血压时序关联级联特征非线性处理单元，用于对所述心率-血压时序关联级联特征向量进行卷积编码和非线性激活处理以得到心率-血压时序关联非线性变换激活向量；A heart rate-blood pressure time series association cascade feature nonlinear processing unit, used for performing convolution coding and nonlinear activation processing on the heart rate-blood pressure time series association cascade feature vector to obtain a heart rate-blood pressure time series association nonlinear transformation activation vector;
患者心率时序关联优化单元，用于将所述心率-血压时序关联非线性变换激活向量输入sigmoid函数以得到心率-血压时序关联第一权重向量后，将所述心率-血压时序关联第一权重向量与所述患者心率时序关联特征向量进行按位置点乘以得到患者心率时序关联优化特征向量；A patient heart rate timing association optimization unit, used for inputting the heart rate-blood pressure timing association nonlinear transformation activation vector into a sigmoid function to obtain a heart rate-blood pressure timing association first weight vector, and then performing position point multiplication of the heart rate-blood pressure timing association first weight vector and the patient heart rate timing association feature vector to obtain a patient heart rate timing association optimization feature vector;
心率-血压时序关联非线性变换差值激活单元，用于计算单位向量与所述心率-血压时序关联非线性变换激活向量的按位置减法以得到心率-血压时序关联非线性变换差值激活向量；A heart rate-blood pressure time series associated nonlinear transformation difference activation unit, used for calculating the positional subtraction between a unit vector and the heart rate-blood pressure time series associated nonlinear transformation activation vector to obtain a heart rate-blood pressure time series associated nonlinear transformation difference activation vector;
患者血压时序关联优化单元，用于将所述心率-血压时序关联非线性变换差值激活向量输入sigmoid函数以得到心率-血压时序关联第二权重向量后，将所述心率-血压时序关联第二权重向量与所述患者血压时序关联特征向量进行按位置点乘以得到患者血压时序关联优化特征向量；A patient blood pressure timing association optimization unit, for inputting the heart rate-blood pressure timing association nonlinear transformation difference activation vector into a sigmoid function to obtain a heart rate-blood pressure timing association second weight vector, and then performing position point multiplication of the heart rate-blood pressure timing association second weight vector and the patient blood pressure timing association feature vector to obtain a patient blood pressure timing association optimization feature vector;
多模态生命体征时序语义动态交互融合单元，用于计算所述患者心率时序关联优化特征向量和所述患者血压时序关联优化特征向量的按位置加法以得到所述多模态生命体征时序语义动态交互融合特征向量。The multimodal vital sign time series semantic dynamic interaction fusion unit is used to calculate the positional addition of the patient's heart rate time series associated optimized feature vector and the patient's blood pressure time series associated optimized feature vector to obtain the multimodal vital sign time series semantic dynamic interaction fusion feature vector.
在上述多发性骨髓瘤心脏淀粉样变患者心理状态分析系统中，所述心率-血压时序关联级联特征非线性处理单元，用于：In the above-mentioned psychological state analysis system for patients with multiple myeloma cardiac amyloidosis, the heart rate-blood pressure time series correlation cascade feature nonlinear processing unit is used to:
对所述心率-血压时序关联级联特征向量进行一维卷积编码以得到心率-血压时序关联级联卷积编码特征向量；Performing one-dimensional convolution coding on the heart rate-blood pressure time series associated cascade feature vector to obtain a heart rate-blood pressure time series associated cascade convolution coding feature vector;
将所述心率-血压时序关联级联卷积编码特征向量与参数矩阵进行向量矩阵相乘以得到心率-血压时序关联级联参数向量后，将所述心率-血压时序关联级联参数向量与偏置向量进行按位置相加以得到心率-血压时序关联级联偏置特征向量；After vector-matrix multiplication of the heart rate-blood pressure timing association cascade convolution coding feature vector and the parameter matrix to obtain the heart rate-blood pressure timing association cascade parameter vector, the heart rate-blood pressure timing association cascade parameter vector and the bias vector are added by position to obtain the heart rate-blood pressure timing association cascade bias feature vector;
将所述心率-血压时序关联级联偏置特征向量输入SiLU激活函数进行非线性激活处理以得到所述心率-血压时序关联非线性变换激活向量。The heart rate-blood pressure time series association cascade bias feature vector is input into the SiLU activation function for nonlinear activation processing to obtain the heart rate-blood pressure time series association nonlinear transformation activation vector.
在上述多发性骨髓瘤心脏淀粉样变患者心理状态分析系统中，所述超声心动图语义特征提取模块，用于：将所述超声心动图通过基于空洞卷积神经网络的超声心动图语义特征提取器以得到所述超声心动图语义特征向量。In the above-mentioned psychological state analysis system for patients with cardiac amyloidosis of multiple myeloma, the echocardiogram semantic feature extraction module is used to: pass the echocardiogram through an echocardiogram semantic feature extractor based on a hollow convolutional neural network to obtain the echocardiogram semantic feature vector.
在上述多发性骨髓瘤心脏淀粉样变患者心理状态分析系统中，所述生命体征时序-超声心动图特征交互模块，包括：In the above-mentioned psychological state analysis system for patients with multiple myeloma cardiac amyloidosis, the vital sign time series-echocardiogram feature interaction module includes:
生命体征-超声心动图语义逐位置响应单元，用于计算所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量之间的逐位置响应以得到生命体征-超声心动图语义逐位置响应特征向量；A vital sign-echocardiogram semantic position-by-position response unit, used for calculating the position-by-position response between the multimodal vital sign time series semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector to obtain a vital sign-echocardiogram semantic position-by-position response feature vector;
归一化单元，用于将所述生命体征-超声心动图语义逐位置响应特征向量输入Softmax函数进行归一化处理以得到归一化生命体征-超声心动图语义逐位置响应特征向量；A normalization unit, used for inputting the vital sign-echocardiogram semantic position-by-position response feature vector into a Softmax function for normalization processing to obtain a normalized vital sign-echocardiogram semantic position-by-position response feature vector;
生命体征-超声心动图语义响应权重掩码计算单元，用于将所述归一化生命体征-超声心动图语义逐位置响应特征向量输入可学习的门控函数以得到生命体征-超声心动图语义响应筛选权重掩码向量；A vital sign-echocardiogram semantic response weight mask calculation unit, used for inputting the normalized vital sign-echocardiogram semantic position-by-position response feature vector into a learnable gating function to obtain a vital sign-echocardiogram semantic response screening weight mask vector;
可区分权重掩码计算单元，用于计算所述生命体征-超声心动图语义响应筛选权重掩码向量与所述归一化生命体征-超声心动图语义逐位置响应特征向量之间的按位置点乘以得到生命体征-超声心动图语义逐位置响应可区分权重掩码向量；A distinguishable weight mask calculation unit, used for calculating the position-wise multiplication between the vital sign-echocardiogram semantic response screening weight mask vector and the normalized vital sign-echocardiogram semantic position-by-position response feature vector to obtain a vital sign-echocardiogram semantic position-by-position response distinguishable weight mask vector;
超声心动语义影响下患者心理状态时序表征单元，用于计算所述生命体征-超声心动图语义逐位置响应可区分权重掩码向量与所述生命体征-超声心动图语义逐位置响应特征向量之间的按位置点乘以得到所述超声心动语义影响下患者心理状态时序表征向量。The patient's psychological state temporal representation unit under the influence of ultrasonic cardiology semantics is used to calculate the distinguishable weight mask vector of the vital sign-echocardiography semantic position-by-position response and the vital sign-echocardiography semantic position-by-position response feature vector to obtain the patient's psychological state temporal representation vector under the influence of ultrasonic cardiology semantics.
在上述多发性骨髓瘤心脏淀粉样变患者心理状态分析系统中，所述生命体征-超声心动图语义响应权重掩码计算单元，用于：In the above-mentioned psychological state analysis system for patients with multiple myeloma cardiac amyloidosis, the vital sign-echocardiogram semantic response weight mask calculation unit is used to:
以所述归一化生命体征-超声心动图语义逐位置响应特征向量中的各个位置特征值的负数作为自然常数的指数以计算按位置的以e为底的指数函数值以得到归一化生命体征-超声心动图语义逐位置响应类支持向量；Using the negative number of each position feature value in the normalized vital sign-echocardiogram semantic position-by-position response feature vector as the exponent of the natural constant to calculate the position-based exponential function value with e as the base to obtain the normalized vital sign-echocardiogram semantic position-by-position response class support vector;
计算所述归一化生命体征-超声心动图语义逐位置响应类支持向量中各个位置特征值与常数一之和的倒数以得到所述生命体征-超声心动图语义响应筛选权重掩码向量。The inverse of the sum of the characteristic value of each position in the normalized vital sign-echocardiogram semantic position-by-position response class support vector and a constant one is calculated to obtain the vital sign-echocardiogram semantic response screening weight mask vector.
在上述多发性骨髓瘤心脏淀粉样变患者心理状态分析系统中，所述心理状态等级评估模块，用于：将所述超声心动语义影响下患者心理状态时序表征向量通过基于分类器的患者心理状态检测器以得到所述检测结果，所述检测结果用于表示被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级标签。In the above-mentioned psychological state analysis system for patients with cardiac amyloidosis of multiple myeloma, the psychological state level assessment module is used to: pass the patient's psychological state time series representation vector under the influence of ultrasonic cardiac semantics through a classifier-based patient psychological state detector to obtain the detection result, and the detection result is used to represent the psychological state level label of the monitored multiple myeloma cardiac amyloidosis patient.
根据本申请的另一个方面，提供了一种多发性骨髓瘤心脏淀粉样变患者心理状态分析方法，其包括：According to another aspect of the present application, a method for analyzing the psychological state of a patient with cardiac amyloidosis of multiple myeloma is provided, comprising:
获取被监测多发性骨髓瘤心脏淀粉样变患者的超声心动图；Obtain echocardiograms of patients being monitored for cardiac amyloidosis in multiple myeloma;
获取由传感器组采集的所述被监测多发性骨髓瘤心脏淀粉样变患者的生理数据的时间序列，其中，所述生理数据包括心率值和血压值；Acquire a time series of physiological data of the monitored multiple myeloma cardiac amyloidosis patient collected by the sensor group, wherein the physiological data includes a heart rate value and a blood pressure value;
将所述生理数据的时间序列进行数据规整以得到患者心率时序输入向量和患者血压时序输入向量后通过患者生命体征时序特征提取器以得到患者心率时序关联特征向量和患者血压时序关联特征向量；Regularizing the time series of the physiological data to obtain a patient heart rate time series input vector and a patient blood pressure time series input vector, and then passing the vectors through a patient vital sign time series feature extractor to obtain a patient heart rate time series associated feature vector and a patient blood pressure time series associated feature vector;
将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量通过基于门控响应机制的生命体征多模态参数时序动态交互融合模块以得到多模态生命体征时序语义动态交互融合特征向量；The patient's heart rate time series associated feature vector and the patient's blood pressure time series associated feature vector are passed through a vital sign multimodal parameter time series dynamic interactive fusion module based on a gated response mechanism to obtain a multimodal vital sign time series semantic dynamic interactive fusion feature vector;
将所述超声心动图通过超声心动图语义特征提取器以得到超声心动图语义特征向量；Passing the echocardiogram through an echocardiogram semantic feature extractor to obtain an echocardiogram semantic feature vector;
将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器以得到超声心动语义影响下患者心理状态时序表征向量作为超声心动语义影响下患者心理状态时序表征特征；The multimodal vital sign time series semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector are fused through a vital sign-echocardiogram semantic dynamic response interaction encoder based on an adaptive response feature differentiation network to obtain a patient psychological state time series representation vector under the influence of echocardiogram semantics as a patient psychological state time series representation feature under the influence of echocardiogram semantics;
基于所述超声心动语义影响下患者心理状态时序表征特征，得到检测结果，所述检测结果用于表示被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级标签。Based on the temporal characterization characteristics of the patient's psychological state under the influence of the ultrasonic cardiology semantics, a detection result is obtained, and the detection result is used to represent the psychological state level label of the monitored multiple myeloma cardiac amyloidosis patient.
与现有技术相比，本申请提供的一种多发性骨髓瘤心脏淀粉样变患者心理状态分析系统及方法，其通过采用基于深度学习的图像分析和数据处理技术，来对超声心动图和生理数据分别进行语义特征提取和时序关联交互分析，以此根据所述超声心动图语义特征影响下生命体征在时序语义上的特征表示来自动地评估被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级。这样，通过分析超声心动图和生理数据，系统能够捕获与心理状态相关的细微生理变化，这些变化可能在传统评估方法中被忽略。并且利用传感器实时采集患者的心率和血压数据，能够连续监测患者的心理状态变化，及时响应患者的心理需求，从而可以从早期的生理变化中识别出潜在的心理问题，为早期干预提供依据。Compared with the prior art, the present application provides a system and method for analyzing the psychological state of patients with cardiac amyloidosis of multiple myeloma, which uses image analysis and data processing technology based on deep learning to extract semantic features and perform time-series correlation interaction analysis on echocardiograms and physiological data, respectively, so as to automatically evaluate the psychological state level of the monitored patients with cardiac amyloidosis of multiple myeloma according to the feature representation of vital signs in time-series semantics under the influence of the semantic features of the echocardiogram. In this way, by analyzing the echocardiogram and physiological data, the system can capture subtle physiological changes related to the psychological state, which may be ignored in traditional evaluation methods. And by using sensors to collect the patient's heart rate and blood pressure data in real time, it is possible to continuously monitor the patient's psychological state changes and respond to the patient's psychological needs in a timely manner, so that potential psychological problems can be identified from early physiological changes, providing a basis for early intervention.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
通过结合附图对本申请实施例进行更详细的描述，本申请的上述以及其他目的、特征和优势将变得更加明显。附图用来提供对本申请实施例的进一步理解，并且构成说明书的一部分，与本申请实施例一起用于解释本申请，并不构成对本申请的限制。在附图中，相同的参考标号通常代表相同部件或步骤。By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
图1为根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析系统的框图；FIG1 is a block diagram of a system for analyzing the psychological state of a patient with cardiac amyloidosis of multiple myeloma according to an embodiment of the present application;
图2为根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析系统的数据流动图；FIG2 is a data flow diagram of a system for analyzing the psychological state of a patient with cardiac amyloidosis of multiple myeloma according to an embodiment of the present application;
图3为根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析方法的流程图。FIG3 is a flow chart of a method for analyzing the psychological state of a patient with cardiac amyloidosis in multiple myeloma according to an embodiment of the present application.
具体实施方式DETAILED DESCRIPTION
下面，将参考附图详细地描述根据本申请的示例实施例。显然，所描述的实施例仅仅是本申请的一部分实施例，而不是本申请的全部实施例，应理解，本申请不受这里描述的示例实施例的限制。Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.
如本申请和权利要求书中所示，除非上下文明确提示例外情形，“一”、“一个”、“一种”和/或“该”等词并非特指单数，也可包括复数。一般说来，术语“包括”与“包含”仅提示包括已明确标识的步骤和元素，而这些步骤和元素不构成一个排它性的罗列，方法或者设备也可能包含其他的步骤或元素。As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and/or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.
虽然本申请对根据本申请的实施例的系统中的某些模块做出了各种引用，然而，任何数量的不同模块可以被使用并运行在用户终端和/或服务器上。所述模块仅是说明性的，并且所述系统和方法的不同方面可以使用不同模块。Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and/or server. The modules are only illustrative, and different aspects of the system and method can use different modules.
本申请中使用了流程图用来说明根据本申请的实施例的系统所执行的操作。应当理解的是，前面或下面操作不一定按照顺序来精确地执行。相反，根据需要，可以按照倒序或同时处理各种步骤。同时，也可以将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.
下面，将参考附图详细地描述根据本申请的示例实施例。显然，所描述的实施例仅仅是本申请的一部分实施例，而不是本申请的全部实施例，应理解，本申请不受这里描述的示例实施例的限制。Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.
传统心理状态评估方法通常依赖于问卷调查或临床访谈，这些方法可能缺乏对细微生理变化的敏感性，如心率变异性或血压的微小波动，而这些细微的变化可能与心理压力相关。此外，由于心理状态可能随时间和环境快速变化，而传统方法可能无法实时捕捉心理状态的快速变化，且无法在没有明显症状的情况下识别潜在的心理问题。因此，期望一种优化的多发性骨髓瘤心脏淀粉样变患者心理状态分析系统，其能够基于患者的超声心动图和生理数据的联合分析来实时捕获并分析患者的心理状态。Traditional methods for assessing mental state usually rely on questionnaires or clinical interviews, which may lack sensitivity to subtle physiological changes, such as small fluctuations in heart rate variability or blood pressure, which may be associated with psychological stress. In addition, because mental state may change rapidly over time and environment, traditional methods may not be able to capture rapid changes in mental state in real time and cannot identify potential psychological problems in the absence of obvious symptoms. Therefore, an optimized system for analyzing the mental state of patients with cardiac amyloidosis in multiple myeloma is desired, which can capture and analyze the patient's mental state in real time based on a joint analysis of the patient's echocardiogram and physiological data.
在本申请的技术方案中，提出了一种多发性骨髓瘤心脏淀粉样变患者心理状态分析系统。图1为根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析系统的框图。图2为根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析系统的数据流动图。如图1和图2所示，根据本申请的实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析系统300，包括：患者超声心动图获取模块310，用于获取被监测多发性骨髓瘤心脏淀粉样变患者的超声心动图；患者生理数据采集模块320，用于获取由传感器组采集的所述被监测多发性骨髓瘤心脏淀粉样变患者的生理数据的时间序列，其中，所述生理数据包括心率值和血压值；生命体征数据时序关联模块330，用于将所述生理数据的时间序列进行数据规整以得到患者心率时序输入向量和患者血压时序输入向量后通过患者生命体征时序特征提取器以得到患者心率时序关联特征向量和患者血压时序关联特征向量；心率-血压时序特征交互融合模块340，用于将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量通过基于门控响应机制的生命体征多模态参数时序动态交互融合模块以得到多模态生命体征时序语义动态交互融合特征向量；超声心动图语义特征提取模块350，用于将所述超声心动图通过超声心动图语义特征提取器以得到超声心动图语义特征向量；生命体征时序-超声心动图特征交互模块360，用于将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器以得到超声心动语义影响下患者心理状态时序表征向量作为超声心动语义影响下患者心理状态时序表征特征；心理状态等级评估模块370，用于基于所述超声心动语义影响下患者心理状态时序表征特征，得到检测结果，所述检测结果用于表示被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级标签。In the technical solution of the present application, a psychological state analysis system for patients with multiple myeloma cardiac amyloidosis is proposed. FIG1 is a block diagram of a psychological state analysis system for patients with multiple myeloma cardiac amyloidosis according to an embodiment of the present application. FIG2 is a data flow diagram of a psychological state analysis system for patients with multiple myeloma cardiac amyloidosis according to an embodiment of the present application. As shown in FIG1 and FIG2, according to the embodiment of the present application, the psychological state analysis system 300 for patients with cardiac amyloidosis of multiple myeloma includes: a patient echocardiogram acquisition module 310, which is used to acquire the echocardiogram of the monitored patient with cardiac amyloidosis of multiple myeloma; a patient physiological data acquisition module 320, which is used to acquire the time series of physiological data of the monitored patient with cardiac amyloidosis of multiple myeloma acquired by the sensor group, wherein the physiological data includes heart rate value and blood pressure value; a vital sign data time series association module 330, which is used to regularize the time series of the physiological data to obtain the patient heart rate time series input vector and the patient blood pressure time series input vector, and then pass them through the patient vital sign time series feature extractor to obtain the patient heart rate time series association feature vector and the patient blood pressure time series association feature vector; a heart rate-blood pressure time series feature interactive fusion module 340, which is used to combine the patient heart rate time series association feature vector and the patient blood pressure time series association feature vector through a gated response mechanism-based A module for dynamically interacting and fusion of multimodal vital sign parameters in time series to obtain a multimodal vital sign time series semantic dynamic interactive fusion feature vector; an echocardiogram semantic feature extraction module 350, for passing the echocardiogram through an echocardiogram semantic feature extractor to obtain an echocardiogram semantic feature vector; a vital sign time series-echocardiogram feature interaction module 360, for passing the multimodal vital sign time series semantic dynamic interactive fusion feature vector and the echocardiogram semantic feature vector through a vital sign-echocardiogram semantic dynamic response interactive encoder based on an adaptive response feature differentiation network to obtain a patient psychological state time series representation vector under the influence of echocardiography semantics as a patient psychological state time series representation feature under the influence of echocardiography semantics; a psychological state level assessment module 370, for obtaining a test result based on the patient psychological state time series representation feature under the influence of echocardiography semantics, wherein the test result is used to represent the psychological state level label of the monitored multiple myeloma cardiac amyloidosis patient.
特别地，所述患者超声心动图获取模块310和所述患者生理数据采集模块320，用于获取被监测多发性骨髓瘤心脏淀粉样变患者的超声心动图；并获取由传感器组采集的所述被监测多发性骨髓瘤心脏淀粉样变患者的生理数据的时间序列，其中，所述生理数据包括心率值和血压值。应可以理解，所述超声心动图是用于检测超声心动图左心室壁的情况。特别地，心脏淀粉样变患者的超声心动图表现为左心室壁增厚，心肌回声增强，特别是颗粒样回声增强，这是由于淀粉样纤维在心肌间质中的沉积造成的。并且由于考虑到患者的生理指标数据能够反映出患者的心理状态和情绪状态变化，同时临床特征会对患者的情绪和心理产生影响。基于此，在本申请的技术方案中，获取被监测多发性骨髓瘤心脏淀粉样变患者的超声心动图，且获取由传感器组采集的所述被监测多发性骨髓瘤心脏淀粉样变患者的生理数据的时间序列，其中，所述生理数据包括心率值和血压值，这样，能够综合地来分析和评估患者的心理状态，并进行及时干预和治疗。In particular, the patient echocardiogram acquisition module 310 and the patient physiological data acquisition module 320 are used to acquire the echocardiogram of the monitored multiple myeloma cardiac amyloidosis patient; and to acquire the time series of physiological data of the monitored multiple myeloma cardiac amyloidosis patient acquired by the sensor group, wherein the physiological data includes heart rate value and blood pressure value. It should be understood that the echocardiogram is used to detect the condition of the left ventricular wall of the echocardiogram. In particular, the echocardiogram of the patient with cardiac amyloidosis shows thickening of the left ventricular wall, enhanced myocardial echo, especially enhanced granular echo, which is caused by the deposition of amyloid fibers in the myocardial interstitium. And considering that the patient's physiological indicator data can reflect the patient's psychological state and emotional state changes, and the clinical characteristics will have an impact on the patient's emotions and psychology. Based on this, in the technical solution of the present application, an echocardiogram of the monitored multiple myeloma cardiac amyloidosis patient is obtained, and a time series of physiological data of the monitored multiple myeloma cardiac amyloidosis patient collected by a sensor group is obtained, wherein the physiological data includes heart rate values and blood pressure values. In this way, the patient's mental state can be comprehensively analyzed and evaluated, and timely intervention and treatment can be carried out.
特别地，所述生命体征数据时序关联模块330，用于将所述生理数据的时间序列进行数据规整以得到患者心率时序输入向量和患者血压时序输入向量后通过患者生命体征时序特征提取器以得到患者心率时序关联特征向量和患者血压时序关联特征向量。特别地，在本申请的一个具体示例中，首先，将所述生理数据的时间序列按照时间维度和样本维度进行数据规整以得到所述患者心率时序输入向量和患者血压时序输入向量；继而将所述患者心率时序输入向量和所述患者血压时序输入向量通过基于1D_CNN网络的患者生命体征时序特征提取器以得到所述患者心率时序关联特征向量和所述患者血压时序关联特征向量。考虑到所述患者心率时序输入向量和所述患者血压时序输入向量都存在局部时间段内的时序特征信息，基于此，将所述患者心率时序输入向量和所述患者血压时序输入向量通过基于1D_CNN网络的患者生命体征时序特征提取器以捕捉和挖掘出患者心率和血压在局部时间段内的时序动态变化信息和异常波动，得到患者心率时序关联特征向量和患者血压时序关联特征向量。In particular, the vital sign data timing association module 330 is used to perform data regularization on the time series of the physiological data to obtain the patient heart rate timing input vector and the patient blood pressure timing input vector, and then pass them through the patient vital sign timing feature extractor to obtain the patient heart rate timing association feature vector and the patient blood pressure timing association feature vector. In particular, in a specific example of the present application, first, the time series of the physiological data is data regularized according to the time dimension and the sample dimension to obtain the patient heart rate timing input vector and the patient blood pressure timing input vector; then the patient heart rate timing input vector and the patient blood pressure timing input vector are passed through the patient vital sign timing feature extractor based on the 1D_CNN network to obtain the patient heart rate timing association feature vector and the patient blood pressure timing association feature vector. Taking into account that both the patient's heart rate timing input vector and the patient's blood pressure timing input vector contain timing feature information within a local time period, based on this, the patient's heart rate timing input vector and the patient's blood pressure timing input vector are passed through a patient vital sign timing feature extractor based on a 1D_CNN network to capture and mine out the timing dynamic change information and abnormal fluctuations of the patient's heart rate and blood pressure within a local time period, and obtain the patient's heart rate timing association feature vector and the patient's blood pressure timing association feature vector.
特别地，所述心率-血压时序特征交互融合模块340，用于将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量通过基于门控响应机制的生命体征多模态参数时序动态交互融合模块以得到多模态生命体征时序语义动态交互融合特征向量。考虑到所述患者心率时序关联特征向量和所述患者血压时序关联特征向量之间存在时序上的相互交互关系。因此，为了基于两者在时间维度上的交互关系，来对所述患者心率时序关联特征向量和所述患者血压时序关联特征向量进行时序语义交互融合，从而来提供更全面的生理状态描述，在本申请的技术方案中，将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量通过基于门控响应机制的生命体征多模态参数时序动态交互融合模块以得到多模态生命体征时序语义动态交互融合特征向量。In particular, the heart rate-blood pressure time series feature interaction fusion module 340 is used to pass the patient's heart rate time series associated feature vector and the patient's blood pressure time series associated feature vector through a vital sign multimodal parameter time series dynamic interaction fusion module based on a gated response mechanism to obtain a multimodal vital sign time series semantic dynamic interaction fusion feature vector. Considering that there is a temporal mutual interaction relationship between the patient's heart rate time series associated feature vector and the patient's blood pressure time series associated feature vector. Therefore, in order to perform temporal semantic interaction fusion on the patient's heart rate time series associated feature vector and the patient's blood pressure time series associated feature vector based on the interaction relationship between the two in the time dimension, so as to provide a more comprehensive description of the physiological state, in the technical solution of the present application, the patient's heart rate time series associated feature vector and the patient's blood pressure time series associated feature vector are passed through a vital sign multimodal parameter time series dynamic interaction fusion module based on a gated response mechanism to obtain a multimodal vital sign time series semantic dynamic interaction fusion feature vector.
在本申请的实施例中，将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量通过基于门控响应机制的生命体征多模态参数时序动态交互融合模块以得到多模态生命体征时序语义动态交互融合特征向量，包括：先将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量进行级联处理以得到心率-血压时序关联级联特征向量；通过将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量进行级联处理，来更充分地表达心率和血压这两个不同生理参数的在时序上的特征，捕获它们之间可能存在的复杂时序相互交互作用，为后续的门控响应提供了丰富的输入。接着，对所述心率-血压时序关联级联特征向量进行卷积编码和非线性激活处理以得到心率-血压时序关联非线性变换激活向量；这里，对级联后得到的特征进行一维卷积编码和非线性激活处理来提取和增强关于心率和血压时序融合特征中的局部模式和趋势，并捕捉关键的局部时序信息，得到心率-血压时序关联非线性变换激活向量。然后，将所述心率-血压时序关联非线性变换激活向量输入sigmoid函数以得到心率-血压时序关联第一权重向量后，将所述心率-血压时序关联第一权重向量与所述患者心率时序关联特征向量进行按位置点乘以得到患者心率时序关联优化特征向量；再计算单位向量与所述心率-血压时序关联非线性变换激活向量的按位置减法以得到心率-血压时序关联非线性变换差值激活向量；将所述心率-血压时序关联非线性变换差值激活向量输入sigmoid函数以得到心率-血压时序关联第二权重向量后，将所述心率-血压时序关联第二权重向量与所述患者血压时序关联特征向量进行按位置点乘以得到患者血压时序关联优化特征向量；随后计算所述患者心率时序关联优化特征向量和所述患者血压时序关联优化特征向量的按位置加法以得到所述多模态生命体征时序语义动态交互融合特征向量。特别地，这里使用SiLU函数来进行激活处理，也就是SiLU通过sigmoid函数对输入向量进行自适应门控，这意味着它能够自动调节激活输出的幅度，从而提高网络的非线性表达能力。最后，利用该非线性变换激活向量来对所述患者心率时序关联特征向量和所述患者血压时序关联特征向量进行加权融合，以动态地捕捉这些时序数据中的关键的时序动态交互特性，以获得更全面的多模态生命体征时序语义动态交互融合特征表示。In an embodiment of the present application, the patient's heart rate timing association feature vector and the patient's blood pressure timing association feature vector are passed through a vital sign multimodal parameter timing dynamic interaction fusion module based on a gated response mechanism to obtain a multimodal vital sign timing semantic dynamic interaction fusion feature vector, including: first cascading the patient's heart rate timing association feature vector and the patient's blood pressure timing association feature vector to obtain a heart rate-blood pressure timing association cascade feature vector; cascading the patient's heart rate timing association feature vector and the patient's blood pressure timing association feature vector to more fully express the timing characteristics of the two different physiological parameters of heart rate and blood pressure, capture the complex timing interactions that may exist between them, and provide rich input for subsequent gated responses. Next, the heart rate-blood pressure time series association cascade feature vector is convolutionally encoded and nonlinearly activated to obtain a heart rate-blood pressure time series association nonlinear transformation activation vector; here, the features obtained after the cascade are subjected to one-dimensional convolutional encoding and nonlinear activation processing to extract and enhance local patterns and trends in the heart rate and blood pressure time series fusion features, and capture key local time series information to obtain a heart rate-blood pressure time series association nonlinear transformation activation vector. Then, the heart rate-blood pressure time series association nonlinear transformation activation vector is input into the sigmoid function to obtain the heart rate-blood pressure time series association first weight vector, and the heart rate-blood pressure time series association first weight vector is multiplied by the patient's heart rate time series association feature vector to obtain the patient's heart rate time series association optimized feature vector; then the positional subtraction of the unit vector and the heart rate-blood pressure time series association nonlinear transformation activation vector is calculated to obtain the heart rate-blood pressure time series association nonlinear transformation difference activation vector; the heart rate-blood pressure time series association nonlinear transformation difference activation vector is input into the sigmoid function to obtain the heart rate-blood pressure time series association second weight vector, and the heart rate-blood pressure time series association second weight vector is multiplied by the patient's blood pressure time series association feature vector to obtain the patient's blood pressure time series association optimized feature vector; then the positional addition of the patient's heart rate time series association optimized feature vector and the patient's blood pressure time series association optimized feature vector is calculated to obtain the multimodal vital sign time series semantic dynamic interaction fusion feature vector. In particular, the SiLU function is used here for activation processing, that is, SiLU adaptively gates the input vector through the sigmoid function, which means that it can automatically adjust the amplitude of the activation output, thereby improving the nonlinear expression ability of the network. Finally, the nonlinear transformation activation vector is used to perform weighted fusion on the patient's heart rate time series associated feature vector and the patient's blood pressure time series associated feature vector to dynamically capture the key time series dynamic interaction characteristics in these time series data, so as to obtain a more comprehensive multimodal vital sign time series semantic dynamic interaction fusion feature representation.
其中，对所述心率-血压时序关联级联特征向量进行卷积编码和非线性激活处理以得到心率-血压时序关联非线性变换激活向量的过程包括：对所述心率-血压时序关联级联特征向量进行一维卷积编码以得到心率-血压时序关联级联卷积编码特征向量；将所述心率-血压时序关联级联卷积编码特征向量与参数矩阵进行向量矩阵相乘以得到心率-血压时序关联级联参数向量后，将所述心率-血压时序关联级联参数向量与偏置向量进行按位置相加以得到心率-血压时序关联级联偏置特征向量；将所述心率-血压时序关联级联偏置特征向量输入SiLU激活函数进行非线性激活处理以得到所述心率-血压时序关联非线性变换激活向量。Among them, the process of performing convolution coding and nonlinear activation processing on the heart rate-blood pressure timing association cascade feature vector to obtain the heart rate-blood pressure timing association nonlinear transformation activation vector includes: performing one-dimensional convolution coding on the heart rate-blood pressure timing association cascade feature vector to obtain the heart rate-blood pressure timing association cascade convolution coding feature vector; after vector-matrix multiplication of the heart rate-blood pressure timing association cascade convolution coding feature vector and the parameter matrix to obtain the heart rate-blood pressure timing association cascade parameter vector, the heart rate-blood pressure timing association cascade parameter vector and the bias vector are added by position to obtain the heart rate-blood pressure timing association cascade bias feature vector; the heart rate-blood pressure timing association cascade bias feature vector is input into the SiLU activation function for nonlinear activation processing to obtain the heart rate-blood pressure timing association nonlinear transformation activation vector.
综上，在上述实施例中，将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量通过基于门控响应机制的生命体征多模态参数时序动态交互融合模块以得到多模态生命体征时序语义动态交互融合特征向量，包括：将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量通过基于门控响应机制的生命体征多模态参数时序动态交互融合模块，以如下动态交互融合公式进行处理以得到所述多模态生命体征时序语义动态交互融合特征向量；其中，所述动态交互融合公式为：In summary, in the above embodiment, the patient's heart rate timing-related feature vector and the patient's blood pressure timing-related feature vector are processed through a vital sign multimodal parameter timing dynamic interaction fusion module based on a gated response mechanism to obtain a multimodal vital sign timing semantic dynamic interaction fusion feature vector, including: the patient's heart rate timing-related feature vector and the patient's blood pressure timing-related feature vector are processed through a vital sign multimodal parameter timing dynamic interaction fusion module based on a gated response mechanism with the following dynamic interaction fusion formula to obtain the multimodal vital sign timing semantic dynamic interaction fusion feature vector; wherein the dynamic interaction fusion formula is:
； ;
； ;
其中，和分别是所述患者心率时序关联特征向量和所述患者血压时序关联特征向量，是级联处理，是对向量进行一维卷积编码，是所述参数矩阵，是所述偏置向量，是函数，是所述心率-血压时序关联非线性变换激活向量，为单位向量，是函数，为按位置点乘，是所述多模态生命体征时序语义动态交互融合特征向量。in, and are respectively the patient's heart rate time series associated feature vector and the patient's blood pressure time series associated feature vector, It is cascade processing. It is a one-dimensional convolutional encoding of the vector. is the parameter matrix, is the bias vector, yes function, is the nonlinear transformation activation vector of the heart rate-blood pressure temporal association, is a unit vector, yes function, To multiply by position, It is the multimodal vital sign temporal semantic dynamic interactive fusion feature vector.
特别地，所述超声心动图语义特征提取模块350，用于将所述超声心动图通过超声心动图语义特征提取器以得到超声心动图语义特征向量。考虑到所述超声心动图中包含了关于心脏组织的隐含语义特征信息，而传统的超声心动图语义特征提取器中卷积网络的感受野较小，对于较大范围的心脏组织和结构的捕捉能力较弱，基于此，在本申请的技术方案中，将所述超声心动图通过基于空洞卷积神经网络的超声心动图语义特征提取器以捕捉和提取出更加丰富和细节的超心动图语义特征信息，得到超声心动图语义特征向量。值得一提的是，空洞卷积神经网络是一种卷积神经网络（CNN），它使用空洞卷积操作来扩大感受野，同时保持空间分辨率。In particular, the echocardiogram semantic feature extraction module 350 is used to pass the echocardiogram through an echocardiogram semantic feature extractor to obtain an echocardiogram semantic feature vector. Considering that the echocardiogram contains implicit semantic feature information about cardiac tissue, and the receptive field of the convolutional network in the traditional echocardiogram semantic feature extractor is small, and the ability to capture a larger range of cardiac tissue and structure is weak. Based on this, in the technical solution of the present application, the echocardiogram is passed through an echocardiogram semantic feature extractor based on a dilated convolutional neural network to capture and extract richer and more detailed echocardiogram semantic feature information to obtain an echocardiogram semantic feature vector. It is worth mentioning that the dilated convolutional neural network is a convolutional neural network (CNN) that uses a dilated convolution operation to expand the receptive field while maintaining spatial resolution.
特别地，所述生命体征时序-超声心动图特征交互模块360，用于将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器以得到超声心动语义影响下患者心理状态时序表征向量作为超声心动语义影响下患者心理状态时序表征特征。考虑到所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量在局部语义上存在着相互关联影响关系。因此，为了进一步细化和强化多模态生命体征时序语义动态交互融合特征和所述超声心动图语义特征之间的逐粒度语义交互作用，以此整合来自不同数据源（生理时序数据和超声心动图图像数据）的语义特征，以获得更全面的患者信息，在本申请的技术方案中，将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器以得到超声心动语义影响下患者心理状态时序表征向量。应可以理解，所述基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器通过计算所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量在每个对应位置之间的语义响应，以更加细致地来揭示两者之间在语义上的潜在交互作用得到逐位置响应特征向量。接着对逐位置响应特征向量进行归一化处理和门控自适应选择得到筛选权重掩码向量，以此动态调整了特征位置的重要性。并利用该筛选权重掩码向量对归一化处理后的特征进行加权强调更有代表性的特征，最后，将该加权后的向量和逐位置响应特征向量进行相乘生成一个综合了多模态信息和语义特征重要性的超声心动语义影响下患者心理状态时序表征向量，这为准确地评估和诊断多发性骨髓瘤心脏淀粉样变患者的心理状态提供了一个强有力的信息基础。In particular, the vital sign timing-echocardiogram feature interaction module 360 is used to obtain the patient's psychological state temporal representation vector under the influence of ultrasound cardiology semantics as the patient's psychological state temporal representation feature under the influence of ultrasound cardiology semantics by using the multimodal vital sign timing semantic dynamic interaction fusion feature vector and the ultrasound cardiography semantic feature vector through a vital sign-echocardiogram semantic dynamic response interaction encoder based on an adaptive response feature distinction network. Considering that the multimodal vital sign timing semantic dynamic interaction fusion feature vector and the ultrasound cardiography semantic feature vector have a mutual correlation and influence relationship in local semantics. Therefore, in order to further refine and strengthen the granular semantic interaction between the multimodal vital sign time series semantic dynamic interaction fusion feature and the echocardiography semantic feature, so as to integrate the semantic features from different data sources (physiological time series data and echocardiography image data) to obtain more comprehensive patient information, in the technical solution of the present application, the multimodal vital sign time series semantic dynamic interaction fusion feature vector and the echocardiography semantic feature vector are passed through the vital sign-echocardiography semantic dynamic response interaction encoder based on the adaptive response feature differentiation network to obtain the patient's psychological state time series representation vector under the influence of echocardiography semantics. It should be understood that the vital sign-echocardiography semantic dynamic response interaction encoder based on the adaptive response feature differentiation network calculates the semantic response between the multimodal vital sign time series semantic dynamic interaction fusion feature vector and the echocardiography semantic feature vector at each corresponding position, so as to more carefully reveal the potential interaction between the two in semantics to obtain the position-by-position response feature vector. Then, the position-by-position response feature vector is normalized and gated adaptively selected to obtain the screening weight mask vector, so as to dynamically adjust the importance of the feature position. The screening weight mask vector is used to weight the normalized features to emphasize the more representative features. Finally, the weighted vector is multiplied with the position-by-position response feature vector to generate a temporal representation vector of the patient's psychological state under the influence of ultrasonic cardiology semantics that integrates the importance of multimodal information and semantic features. This provides a strong information basis for accurately evaluating and diagnosing the psychological state of patients with multiple myeloma cardiac amyloidosis.
在本申请的实施例中，将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器以得到超声心动语义影响下患者心理状态时序表征向量作为超声心动语义影响下患者心理状态时序表征特征，包括：计算所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量之间的逐位置响应以得到生命体征-超声心动图语义逐位置响应特征向量；将所述生命体征-超声心动图语义逐位置响应特征向量输入Softmax函数进行归一化处理以得到归一化生命体征-超声心动图语义逐位置响应特征向量；生命体征-超声心动图语义响应权重掩码计算单元，用于将所述归一化生命体征-超声心动图语义逐位置响应特征向量输入可学习的门控函数以得到生命体征-超声心动图语义响应筛选权重掩码向量；计算所述生命体征-超声心动图语义响应筛选权重掩码向量与所述归一化生命体征-超声心动图语义逐位置响应特征向量之间的按位置点乘以得到生命体征-超声心动图语义逐位置响应可区分权重掩码向量；计算所述生命体征-超声心动图语义逐位置响应可区分权重掩码向量与所述生命体征-超声心动图语义逐位置响应特征向量之间的按位置点乘以得到所述超声心动语义影响下患者心理状态时序表征向量。In an embodiment of the present application, the multimodal vital sign temporal semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector are transformed into a vital sign-echocardiogram semantic dynamic response interaction encoder based on an adaptive response feature differentiation network to obtain a patient psychological state temporal representation vector under the influence of echocardiogram semantics as a patient psychological state temporal representation feature under the influence of echocardiogram semantics, including: calculating the position-by-position response between the multimodal vital sign temporal semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector to obtain a vital sign-echocardiogram semantic position-by-position response feature vector; inputting the vital sign-echocardiogram semantic position-by-position response feature vector into a Softmax function for normalization processing to obtain a normalized vital sign-echocardiogram semantic position-by-position response feature vector. Feature vector; a vital sign-echocardiogram semantic response weight mask calculation unit, used to input the normalized vital sign-echocardiogram semantic position-by-position response feature vector into a learnable gating function to obtain a vital sign-echocardiogram semantic response screening weight mask vector; calculate the position point multiplication between the vital sign-echocardiogram semantic response screening weight mask vector and the normalized vital sign-echocardiogram semantic position-by-position response feature vector to obtain a vital sign-echocardiogram semantic position-by-position response distinguishable weight mask vector; calculate the position point multiplication between the vital sign-echocardiogram semantic position-by-position response distinguishable weight mask vector and the vital sign-echocardiogram semantic position-by-position response feature vector to obtain a patient's psychological state temporal representation vector under the influence of echocardiography semantics.
其中，将所述归一化生命体征-超声心动图语义逐位置响应特征向量输入可学习的门控函数以得到生命体征-超声心动图语义响应筛选权重掩码向量的过程包括：以所述归一化生命体征-超声心动图语义逐位置响应特征向量中的各个位置特征值的负数作为自然常数的指数以计算按位置的以e为底的指数函数值以得到归一化生命体征-超声心动图语义逐位置响应类支持向量；计算所述归一化生命体征-超声心动图语义逐位置响应类支持向量中各个位置特征值与常数一之和的倒数以得到所述生命体征-超声心动图语义响应筛选权重掩码向量。Among them, the process of inputting the normalized vital sign-echocardiogram semantic position-by-position response feature vector into a learnable gating function to obtain a vital sign-echocardiogram semantic response screening weight mask vector includes: using the negative number of each position feature value in the normalized vital sign-echocardiogram semantic position-by-position response feature vector as the exponent of a natural constant to calculate the position-based exponential function value with e as the base to obtain a normalized vital sign-echocardiogram semantic position-by-position response class support vector; calculating the inverse of the sum of each position feature value in the normalized vital sign-echocardiogram semantic position-by-position response class support vector and a constant one to obtain the vital sign-echocardiogram semantic response screening weight mask vector.
综上，在上述实施例中，将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器以得到超声心动语义影响下患者心理状态时序表征向量作为超声心动语义影响下患者心理状态时序表征特征，包括：将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器，以如下动态响应交互公式进行处理以得到所述超声心动语义影响下患者心理状态时序表征向量；其中，所述动态响应交互公式为:In summary, in the above embodiment, the multimodal vital sign time series semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector are processed through a vital sign-echocardiogram semantic dynamic response interaction encoder based on an adaptive response feature differentiation network to obtain a patient psychological state time series representation vector under the influence of echocardiogram semantics as a patient psychological state time series representation feature under the influence of echocardiogram semantics, including: the multimodal vital sign time series semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector are processed through a vital sign-echocardiogram semantic dynamic response interaction encoder based on an adaptive response feature differentiation network with the following dynamic response interaction formula to obtain the patient psychological state time series representation vector under the influence of echocardiogram semantics; wherein the dynamic response interaction formula is:
； ;
； ;
； ;
； ;
； ;
其中，和分别是所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量，表示所述生命体征-超声心动图语义逐位置响应特征向量，是函数，是所述归一化生命体征-超声心动图语义逐位置响应特征向量，是以自然常数为底的指数函数，是所述生命体征-超声心动图语义响应筛选权重掩码向量，是按位置点乘，是所述生命体征-超声心动图语义逐位置响应可区分权重掩码向量，是所述超声心动语义影响下患者心理状态时序表征向量。in, and They are respectively the multimodal vital sign time series semantic dynamic interactive fusion feature vector and the echocardiogram semantic feature vector, represents the vital sign-echocardiogram semantic position-by-position response feature vector, yes function, is the normalized vital sign-echocardiogram semantic position-by-position response feature vector, The natural constant The exponential function with base , is the vital sign-echocardiogram semantic response screening weight mask vector, It is the point multiplication by position. is the vital sign-echocardiogram semantic position-by-position response distinguishable weight mask vector, It is a temporal representation vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics.
特别地，所述心理状态等级评估模块370，用于基于所述超声心动语义影响下患者心理状态时序表征特征，得到检测结果，所述检测结果用于表示被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级标签。在本申请的具体示例中，将所述超声心动语义影响下患者心理状态时序表征向量通过基于分类器的患者心理状态检测器以得到所述检测结果，所述检测结果用于表示被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级标签。也就是，利用所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量进行语义动态交互得到的超声心动语义影响下患者心理状态时序表征特征进行分类处理，以此来自动地评估被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级。这样，通过分析超声心动图和生理数据，系统能够捕获与心理状态相关的细微生理变化，这些变化可能在传统评估方法中被忽略。并且利用传感器实时采集患者的心率和血压数据，能够连续监测患者的心理状态变化，及时响应患者的心理需求，从而可以从早期的生理变化中识别出潜在的心理问题，为早期干预提供依据。In particular, the psychological state level assessment module 370 is used to obtain a test result based on the temporal characterization feature of the patient's psychological state under the influence of the ultrasonic cardiography semantics, and the test result is used to represent the psychological state level label of the monitored multiple myeloma cardiac amyloidosis patient. In a specific example of the present application, the temporal characterization vector of the patient's psychological state under the influence of the ultrasonic cardiography semantics is passed through a patient psychological state detector based on a classifier to obtain the test result, and the test result is used to represent the psychological state level label of the monitored multiple myeloma cardiac amyloidosis patient. That is, the temporal characterization feature of the patient's psychological state under the influence of the ultrasonic cardiography semantics obtained by the semantic dynamic interaction of the multimodal vital sign temporal semantics dynamic interaction fusion feature vector and the ultrasonic cardiography semantic feature vector is classified and processed, so as to automatically evaluate the psychological state level of the monitored multiple myeloma cardiac amyloidosis patient. In this way, by analyzing the ultrasonic cardiography and physiological data, the system can capture subtle physiological changes related to the psychological state, which may be ignored in traditional assessment methods. And by using sensors to collect the patient's heart rate and blood pressure data in real time, it can continuously monitor changes in the patient's mental state and respond to the patient's psychological needs in a timely manner, thereby identifying potential psychological problems from early physiological changes and providing a basis for early intervention.
在一个优选示例中，将所述超声心动语义影响下患者心理状态时序表征向量通过基于分类器的患者心理状态检测器以得到检测结果包括：In a preferred example, the step of passing the patient's psychological state time series representation vector under the influence of ultrasound cardiology semantics through a patient's psychological state detector based on a classifier to obtain a detection result includes:
计算所述超声心动语义影响下患者心理状态时序表征向量的特征均值，并将所述特征均值除以所述超声心动语义影响下患者心理状态时序表征向量的最大特征值与最小特征值之差以获得超声心动语义影响下患者心理状态时序表征分布表征值；Calculating the characteristic mean of the time series representation vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics, and dividing the characteristic mean by the difference between the maximum eigenvalue and the minimum eigenvalue of the time series representation vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics to obtain the distribution representation value of the time series representation of the patient's psychological state under the influence of the ultrasonic cardiology semantics;
将一减去所述超声心动语义影响下患者心理状态时序表征分布表征值后除以所述超声心动语义影响下患者心理状态时序表征分布表征值以得到超声心动语义影响下患者心理状态时序表征分布调制值；The temporal characterization distribution modulation value of the patient's psychological state under the influence of ultrasonic cardiology semantics is obtained by subtracting the temporal characterization distribution characterization value of the patient's psychological state under the influence of ultrasonic cardiology semantics from one and dividing the result by the temporal characterization distribution characterization value of the patient's psychological state under the influence of ultrasonic cardiology semantics to obtain the temporal characterization distribution modulation value of the patient's psychological state under the influence of ultrasonic cardiology semantics;
将所述超声心动语义影响下患者心理状态时序表征向量通过概率化函数进行激活以获得概率化的超声心动语义影响下患者心理状态时序表征向量；Activating the patient's psychological state temporal representation vector under the influence of ultrasonic cardiology semantics through a probabilistic function to obtain a probabilistic patient's psychological state temporal representation vector under the influence of ultrasonic cardiology semantics;
将所述概率化的超声心动语义影响下患者心理状态时序表征向量与所述超声心动语义影响下患者心理状态时序表征分布调制值进行点减后，取绝对值并计算以2为底的对数值的负数以获得概率化的超声心动语义影响下患者心理状态时序表征分布调制信息向量；After performing point subtraction between the probabilistic temporal representation vector of the patient's psychological state under the influence of ultrasonic cardiology semantics and the temporal representation distribution modulation value of the patient's psychological state under the influence of ultrasonic cardiology semantics, taking the absolute value and calculating the negative of the logarithmic value with base 2 to obtain the probabilistic temporal representation distribution modulation information vector of the patient's psychological state under the influence of ultrasonic cardiology semantics;
将所述超声心动语义影响下患者心理状态时序表征分布表征值除以一减去所述概率化的超声心动语义影响下患者心理状态时序表征向量的每个特征值之差后，对所述概率化的超声心动语义影响下患者心理状态时序表征向量的所有特征值求和，并除以所述超声心动语义影响下患者心理状态时序表征向量的长度以获得概率化的超声心动语义影响下患者心理状态时序表征分布调制偏置值；After dividing the distribution representation value of the temporal representation of the patient's psychological state under the influence of ultrasonic cardiology semantics by one minus the difference of each eigenvalue of the temporal representation vector of the patient's psychological state under the influence of the probabilistic ultrasonic cardiology semantics, summing all eigenvalues of the temporal representation vector of the patient's psychological state under the influence of the probabilistic ultrasonic cardiology semantics, and dividing by the length of the temporal representation vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics to obtain the distribution modulation bias value of the temporal representation of the patient's psychological state under the influence of the probabilistic ultrasonic cardiology semantics;
将所述概率化的超声心动语义影响下患者心理状态时序表征分布调制信息向量和所述概率化的超声心动语义影响下患者心理状态时序表征分布调制偏置值与作为超参数的权重的乘积进行点加以获得优化的超声心动语义影响下患者心理状态时序表征向量；The optimized temporal representation vector of the patient's psychological state under the influence of ultrasonic cardiology semantics is obtained by multiplying the product of the probabilistic temporal representation distribution modulation information vector of the patient's psychological state under the influence of ultrasonic cardiology semantics and the probabilistic temporal representation distribution modulation bias value of the patient's psychological state under the influence of ultrasonic cardiology semantics and the weight as a hyperparameter;
将所述优化的超声心动语义影响下患者心理状态时序表征向量通过基于分类器的患者心理状态检测器以得到检测结果。The optimized temporal representation vector of the patient's mental state under the influence of ultrasonic cardiology semantics is passed through a patient mental state detector based on a classifier to obtain a detection result.
其中，所述优化的超声心动语义影响下患者心理状态时序表征向量表示为：The optimized temporal representation vector of the patient's psychological state under the influence of echocardiography semantics is expressed as:
； ;
； ;
其中，是所述概率化的超声心动语义影响下患者心理状态时序表征向量，是所述超声心动语义影响下患者心理状态时序表征分布表征值，按位置点减，表示以2为底的对数函数值，按位置点加，是作为超参数的权重，为所述超声心动语义影响下患者心理状态时序表征向量的长度，是所述概率化的超声心动语义影响下患者心理状态时序表征向量中的各个位置的特征值，是所述超声心动语义影响下患者心理状态时序表征向量的特征均值，和分别是所述超声心动语义影响下患者心理状态时序表征向量的最大特征值与最小特征值，是所述优化的超声心动语义影响下患者心理状态时序表征向量。in, is the temporal representation vector of the patient's psychological state under the influence of the probabilistic echocardiographic semantics, is the temporal representation distribution representation value of the patient's psychological state under the influence of the ultrasonic cardiology semantics, Subtract by position point, represents the logarithmic function value with base 2, Click on the location to add. is the weight as a hyperparameter, is the length of the time series representation vector of the patient's psychological state under the influence of the echocardiography semantics, is the characteristic value of each position in the temporal representation vector of the patient's psychological state under the influence of the probabilistic echocardiography semantics, is the characteristic mean of the temporal representation vector of the patient's psychological state under the influence of the echocardiography semantics, and are respectively the maximum eigenvalue and the minimum eigenvalue of the temporal representation vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics, It is the optimized temporal representation vector of the patient's psychological state under the influence of echocardiographic semantics.
本申请申请人考虑到所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量分别表达被监测多发性骨髓瘤心脏淀粉样变患者的心率值和血压值的局部时序关联动态交互融合特征和超声心动图的图像语义特征，这样，将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器时，得到的所述超声心动语义影响下患者心理状态时序表征向量也会由于生理数据和超声心动图基于各自特征分布模态的特征分布自适应响应差异而引起的语义动态响应交互覆盖不足，从而导致离群类推理映射偏差，影响所述超声心动语义影响下患者心理状态时序表征向量通过基于分类器的患者心理状态检测器得到的检测结果的准确性。The applicant of the present application has taken into account that the multimodal vital sign temporal semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector respectively express the local temporal correlation dynamic interaction fusion features of the heart rate and blood pressure values of the monitored multiple myeloma cardiac amyloidosis patients and the image semantic features of the echocardiogram. In this way, when the multimodal vital sign temporal semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector are passed through a vital sign-echocardiogram semantic dynamic response interaction encoder based on an adaptive response feature differentiation network, the obtained temporal representation vector of the patient's psychological state under the influence of the echocardiogram semantics will also have insufficient semantic dynamic response interaction coverage due to the difference in feature distribution adaptive response of physiological data and echocardiogram based on their respective feature distribution modalities, thereby leading to outlier class reasoning mapping deviation, affecting the accuracy of the detection result obtained by the temporal representation vector of the patient's psychological state under the influence of the echocardiogram semantics through a patient psychological state detector based on a classifier.
基于此，在上述优选示例中，通过所述超声心动语义影响下患者心理状态时序表征向量相对于特征值分布的伯努利概率调制分布来进行所述超声心动语义影响下患者心理状态时序表征向量的基于特征值的概率信息分布规划，并将所述超声心动语义影响下患者心理状态时序表征向量的概率特征整体的概率反向映射作为对于所述超声心动语义影响下患者心理状态时序表征向量的集合映射空间的拓展覆盖，来对所述超声心动语义影响下患者心理状态时序表征向量的直觉的概率信息分布和抽象的概率空间映射进行其间交互路径的独立理解，以通过避免所述超声心动语义影响下患者心理状态时序表征向量的离群特征分布到类概率的反事实推理映射来提升所述优化的超声心动语义影响下患者心理状态时序表征向量通过基于分类器的患者心理状态检测器得到的检测结果的准确性。这样，通过分析超声心动图和生理数据，系统能够捕获与心理状态相关的细微生理变化，这些变化可能在传统评估方法中被忽略。并且利用传感器实时采集患者的心率和血压数据，能够连续监测患者的心理状态变化，及时响应患者的心理需求，从而可以从早期的生理变化中识别出潜在的心理问题，为早期干预提供依据。Based on this, in the above preferred example, the probability information distribution planning based on the eigenvalue of the temporal characterization vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics is performed through the Bernoulli probability modulation distribution of the temporal characterization vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics relative to the eigenvalue distribution, and the probability reverse mapping of the probability characteristics of the temporal characterization vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics is used as the extended coverage of the set mapping space of the temporal characterization vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics, so as to independently understand the interactive path between the intuitive probability information distribution and the abstract probability space mapping of the temporal characterization vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics, so as to improve the accuracy of the detection results obtained by the patient's psychological state detector based on the classifier by avoiding the counterfactual reasoning mapping of the outlier feature distribution of the temporal characterization vector of the patient's psychological state under the influence of the ultrasonic cardiology semantics to the class probability. In this way, by analyzing the echocardiogram and physiological data, the system can capture subtle physiological changes related to the psychological state, which may be ignored in traditional evaluation methods. And by using sensors to collect the patient's heart rate and blood pressure data in real time, it can continuously monitor changes in the patient's mental state and respond to the patient's psychological needs in a timely manner, thereby identifying potential psychological problems from early physiological changes and providing a basis for early intervention.
如上所述，根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析系统300可以实现在各种无线终端中，例如具有多发性骨髓瘤心脏淀粉样变患者心理状态分析算法的服务器等。在一种可能的实现方式中，根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析系统300可以作为一个软件模块和/或硬件模块而集成到无线终端中。例如，该多发性骨髓瘤心脏淀粉样变患者心理状态分析系统300可以是该无线终端的操作系统中的一个软件模块，或者可以是针对于该无线终端所开发的一个应用程序；当然，该多发性骨髓瘤心脏淀粉样变患者心理状态分析系统300同样可以是该无线终端的众多硬件模块之一。As described above, the psychological state analysis system 300 for patients with multiple myeloma cardiac amyloidosis according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a psychological state analysis algorithm for patients with multiple myeloma cardiac amyloidosis. In a possible implementation, the psychological state analysis system 300 for patients with multiple myeloma cardiac amyloidosis according to the embodiment of the present application can be integrated into a wireless terminal as a software module and/or a hardware module. For example, the psychological state analysis system 300 for patients with multiple myeloma cardiac amyloidosis can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the psychological state analysis system 300 for patients with multiple myeloma cardiac amyloidosis can also be one of the many hardware modules of the wireless terminal.
替换地，在另一示例中，该多发性骨髓瘤心脏淀粉样变患者心理状态分析系统300与该无线终端也可以是分立的设备，并且该多发性骨髓瘤心脏淀粉样变患者心理状态分析系统300可以通过有线和/或无线网络连接到该无线终端，并且按照约定的数据格式来传输交互信息。Alternatively, in another example, the multiple myeloma cardiac amyloidosis patient psychological state analysis system 300 and the wireless terminal may also be separate devices, and the multiple myeloma cardiac amyloidosis patient psychological state analysis system 300 may be connected to the wireless terminal via a wired and/or wireless network and transmit interactive information in accordance with an agreed data format.
进一步地，还提供一种多发性骨髓瘤心脏淀粉样变患者心理状态分析方法。Furthermore, a method for analyzing the psychological state of patients with multiple myeloma cardiac amyloidosis is provided.
图3为根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析方法的流程图。如图3所示，根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析方法，包括步骤：S1，获取被监测多发性骨髓瘤心脏淀粉样变患者的超声心动图；S2，获取由传感器组采集的所述被监测多发性骨髓瘤心脏淀粉样变患者的生理数据的时间序列，其中，所述生理数据包括心率值和血压值；S3，将所述生理数据的时间序列进行数据规整以得到患者心率时序输入向量和患者血压时序输入向量后通过患者生命体征时序特征提取器以得到患者心率时序关联特征向量和患者血压时序关联特征向量；S4，将所述患者心率时序关联特征向量和所述患者血压时序关联特征向量通过基于门控响应机制的生命体征多模态参数时序动态交互融合模块以得到多模态生命体征时序语义动态交互融合特征向量；S5，将所述超声心动图通过超声心动图语义特征提取器以得到超声心动图语义特征向量；S6，将所述多模态生命体征时序语义动态交互融合特征向量和所述超声心动图语义特征向量通过基于自适应响应特征区分网络的生命体征-超声心动图语义动态响应交互编码器以得到超声心动语义影响下患者心理状态时序表征向量作为超声心动语义影响下患者心理状态时序表征特征；S7，基于所述超声心动语义影响下患者心理状态时序表征特征，得到检测结果，所述检测结果用于表示被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级标签。FIG3 is a flow chart of a method for analyzing the psychological state of a patient with cardiac amyloidosis of multiple myeloma according to an embodiment of the present application. As shown in FIG3 , the method for analyzing the psychological state of a patient with cardiac amyloidosis of multiple myeloma according to an embodiment of the present application includes the following steps: S1, obtaining an echocardiogram of the monitored patient with cardiac amyloidosis of multiple myeloma; S2, obtaining a time series of physiological data of the monitored patient with cardiac amyloidosis of multiple myeloma collected by a sensor group, wherein the physiological data includes heart rate values and blood pressure values; S3, regularizing the time series of the physiological data to obtain a patient heart rate timing input vector and a patient blood pressure timing input vector, and then passing the data through a patient vital sign timing feature extractor to obtain a patient heart rate timing correlation feature vector and a patient blood pressure timing correlation feature vector; S4, passing the patient heart rate timing correlation feature vector and the patient blood pressure timing correlation feature vector through a vital sign multimodal sensor based on a gated response mechanism. A parameter time series dynamic interaction fusion module is used to obtain a multimodal vital sign time series semantic dynamic interaction fusion feature vector; S5, the echocardiogram is passed through an echocardiogram semantic feature extractor to obtain an echocardiogram semantic feature vector; S6, the multimodal vital sign time series semantic dynamic interaction fusion feature vector and the echocardiogram semantic feature vector are passed through a vital sign-echocardiogram semantic dynamic response interaction encoder based on an adaptive response feature differentiation network to obtain a patient psychological state time series representation vector under the influence of ultrasound semantics as a patient psychological state time series representation feature under the influence of ultrasound semantics; S7, based on the patient psychological state time series representation feature under the influence of ultrasound semantics, a test result is obtained, and the test result is used to represent the psychological state level label of the monitored multiple myeloma cardiac amyloidosis patient.
综上，根据本申请实施例的多发性骨髓瘤心脏淀粉样变患者心理状态分析方法被阐明，其通过采用基于深度学习的图像分析和数据处理技术，来对超声心动图和生理数据分别进行语义特征提取和时序关联交互分析，以此根据所述超声心动图语义特征影响下生命体征在时序语义上的特征表示来自动地评估被监测多发性骨髓瘤心脏淀粉样变患者的心理状态等级。这样，通过分析超声心动图和生理数据，系统能够捕获与心理状态相关的细微生理变化，这些变化可能在传统评估方法中被忽略。并且利用传感器实时采集患者的心率和血压数据，能够连续监测患者的心理状态变化，及时响应患者的心理需求，从而可以从早期的生理变化中识别出潜在的心理问题，为早期干预提供依据。In summary, the psychological state analysis method of patients with cardiac amyloidosis of multiple myeloma according to the embodiment of the present application is explained, which uses image analysis and data processing technology based on deep learning to perform semantic feature extraction and time-series correlation interaction analysis on echocardiogram and physiological data respectively, so as to automatically evaluate the psychological state level of the monitored patients with cardiac amyloidosis of multiple myeloma according to the feature representation of vital signs in time-series semantics under the influence of the semantic features of the echocardiogram. In this way, by analyzing the echocardiogram and physiological data, the system can capture subtle physiological changes related to the psychological state, which may be ignored in traditional evaluation methods. And by using sensors to collect the patient's heart rate and blood pressure data in real time, it is possible to continuously monitor the patient's psychological state changes and respond to the patient's psychological needs in a timely manner, so that potential psychological problems can be identified from early physiological changes, providing a basis for early intervention.
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411103768.1ACN118629596B (en) | 2024-08-13 | 2024-08-13 | Psychological state analysis system and method for patients with multiple myeloma cardiac amyloidosis |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411103768.1ACN118629596B (en) | 2024-08-13 | 2024-08-13 | Psychological state analysis system and method for patients with multiple myeloma cardiac amyloidosis |
| Publication Number | Publication Date |
|---|---|
| CN118629596Atrue CN118629596A (en) | 2024-09-10 |
| CN118629596B CN118629596B (en) | 2024-11-01 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411103768.1AActiveCN118629596B (en) | 2024-08-13 | 2024-08-13 | Psychological state analysis system and method for patients with multiple myeloma cardiac amyloidosis |
| Country | Link |
|---|---|
| CN (1) | CN118629596B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119475104A (en)* | 2025-01-14 | 2025-02-18 | 北京大学第三医院（北京大学第三临床医学院） | HFimpEF prediction method based on electrocardiogram combined with artificial intelligence technology |
| CN119586991A (en)* | 2024-11-12 | 2025-03-11 | 山东大学 | A cuffless continuous blood pressure signal reconstruction method and system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018124309A1 (en)* | 2016-12-30 | 2018-07-05 | Mitsubishi Electric Corporation | Method and system for multi-modal fusion model |
| CN117457229A (en)* | 2023-12-26 | 2024-01-26 | 吉林大学 | Anesthesia depth monitoring system and method based on artificial intelligence |
| CN118471495A (en)* | 2024-05-06 | 2024-08-09 | 中国科学院计算技术研究所 | Perioperative complication risk prediction method based on multi-modal fusion deep learning |
| CN118471543A (en)* | 2024-07-12 | 2024-08-09 | 吉林大学 | Intelligent management system and method for implanting external instruments in operating room |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018124309A1 (en)* | 2016-12-30 | 2018-07-05 | Mitsubishi Electric Corporation | Method and system for multi-modal fusion model |
| CN117457229A (en)* | 2023-12-26 | 2024-01-26 | 吉林大学 | Anesthesia depth monitoring system and method based on artificial intelligence |
| CN118471495A (en)* | 2024-05-06 | 2024-08-09 | 中国科学院计算技术研究所 | Perioperative complication risk prediction method based on multi-modal fusion deep learning |
| CN118471543A (en)* | 2024-07-12 | 2024-08-09 | 吉林大学 | Intelligent management system and method for implanting external instruments in operating room |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119586991A (en)* | 2024-11-12 | 2025-03-11 | 山东大学 | A cuffless continuous blood pressure signal reconstruction method and system |
| CN119475104A (en)* | 2025-01-14 | 2025-02-18 | 北京大学第三医院（北京大学第三临床医学院） | HFimpEF prediction method based on electrocardiogram combined with artificial intelligence technology |
| CN119475104B (en)* | 2025-01-14 | 2025-04-29 | 北京大学第三医院（北京大学第三临床医学院） | HFimpEF prediction method based on electrocardiogram combined with artificial intelligence technology |
| Publication number | Publication date |
|---|---|
| CN118629596B (en) | 2024-11-01 |
| Publication | Publication Date | Title |
|---|---|---|
| Zhao et al. | A hybrid spatio-temporal model for detection and severity rating of Parkinson’s disease from gait data | |
| Bhaskar et al. | Time series classification-based correlational neural network with bidirectional LSTM for automated detection of kidney disease | |
| CN118629596B (en) | Psychological state analysis system and method for patients with multiple myeloma cardiac amyloidosis | |
| Hu et al. | Graph fusion network-based multimodal learning for freezing of gait detection | |
| Verma et al. | An accurate missing data prediction method using LSTM based deep learning for health care | |
| Chen et al. | Edge2Analysis: a novel AIoT platform for atrial fibrillation recognition and detection | |
| Chen et al. | Dynamic illness severity prediction via multi-task rnns for intensive care unit | |
| TWI701681B (en) | Prediction model of atrial fibrillation and prediction system thereof | |
| CN114550907A (en) | Epilepsy detection system | |
| CN118692662A (en) | An audio and video recognition system based on multimodal analysis | |
| Thilakarathne et al. | Artificial intelligence-enabled IoT for health and wellbeing monitoring | |
| Bukhari et al. | A Smart Heart Disease Diagnostic System Using Deep Vanilla LSTM. | |
| Sivasubramaniam et al. | Early detection and prediction of Heart Disease using Wearable devices and Deep Learning algorithms | |
| Ramesh et al. | AI based dynamic prediction model for mobile health application system | |
| Nia et al. | The power of ann-random forest algorithm in human activities recognition using imu data | |
| Onim et al. | Predicting stress in older adults with rnn and lstm from time series sensor data and cortisol | |
| CN111798980A (en) | Complex medical biological signal processing method and device based on deep learning network | |
| CN117976217A (en) | Pain assessment method, device and terminal | |
| Alattab et al. | A smart diseases diagnosis and classification strategy of electronic healthcare application using novel hybrid artificial intelligence approaches | |
| CN117612732A (en) | Multimodal health management plan generation method and system based on deep learning | |
| Asgharnezhad et al. | Improving ppg signal classification with machine learning: The power of a second opinion | |
| US20220262516A1 (en) | Atrial Fibrillation Prediction Model And Prediction System Thereof | |
| Nagarajan et al. | Analysis and evaluation of heart disease using CNN | |
| Sharma et al. | Machine and deep learning approaches for heart disease risk assessment | |
| Rout et al. | Automated Cardiovascular Disease Detection from ECG Images Using Deep Learning |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |