技术领域technical field
本发明涉及心理健康领域、智能传感领域、虚拟现实技术、大数据分析领域和人工智能领域等，具体涉及一种基于虚拟现实技术的心理评估方法。The invention relates to the fields of mental health, intelligent sensing, virtual reality technology, big data analysis, artificial intelligence, etc., and specifically relates to a psychological assessment method based on virtual reality technology.
背景技术Background technique
随着社会经济的快速发展和竞争压力不断加大，各类心理健康问题层出不穷。With the rapid development of social economy and the increasing pressure of competition, various mental health problems emerge in an endless stream.
然而目前我国相应的医疗资源严重不足，我国临床诊断心理问题一般先通过设备检查来排除器质性疾病，然后采用量表评估和医生评估相结合的方式来确诊是否患有心理疾病，医患面对面的诊断过于依赖医生的专业知识及主观意见。鉴于病人普遍具有病耻感，在与医生面对面的交流过程中可能有所隐瞒，这导致医患沟通存在很大困难。However, at present, the corresponding medical resources in our country are seriously insufficient. The clinical diagnosis of psychological problems in our country generally first excludes organic diseases through equipment inspection, and then uses a combination of scale evaluation and doctor evaluation to diagnose whether there is a mental disease. Doctors and patients face to face The diagnosis of the disease relies too much on the professional knowledge and subjective opinions of doctors. Given that patients generally have a sense of stigma, they may conceal something during face-to-face communication with doctors, which leads to great difficulties in doctor-patient communication.
虚拟现实(Virtual Reality，VR)技术在近些年得到大力推广及发展，在医疗领域已有一定的应用，并且在心理疾病治疗方面也进行了尝试。VR技术能构建与真实世界高度相近的虚拟世界，使用者能与虚拟世界进行接近真实的交互，产生身临其境的体验。Virtual reality (Virtual Reality, VR) technology has been vigorously promoted and developed in recent years, and it has been applied in the medical field to a certain extent, and it has also been tried in the treatment of mental diseases. VR technology can build a virtual world that is highly similar to the real world, and users can interact with the virtual world close to reality to create an immersive experience.
将VR技术、智能传感技术、大数据分析技术、人工智能技术与传统心理量表评估相结合，改善传统心理评估方法带来的局限性，其中包括医疗资源的不足、病患的病耻感、评估过程的主观性。通过VR技术构建有效的心理评估场景，为使用者提供自助式心理评估方法，可以节约就医时间，节省医疗资源，增加使用者的舒适度，降低使用者的病耻感，提高使用者的配合度及心理评估的有效性；在评估过程中实时采集使用者生理数据及行为数据，利用智能信息处理技术分析用户实时状态，对受试者问答选项进行智能校正，提高心理评估结果的准确性与客观性；通过标准化问诊过程，可解决传统问诊中量化困难的现状。同时结合大数据技术，进行用户个性化心理特征挖掘与规律总结，提升心理评估的准确性。Combining VR technology, intelligent sensing technology, big data analysis technology, artificial intelligence technology and traditional psychological scale assessment, to improve the limitations of traditional psychological assessment methods, including the shortage of medical resources and the stigma of patients , The subjectivity of the evaluation process. Build an effective psychological assessment scene through VR technology and provide users with self-service psychological assessment methods, which can save time for medical treatment, save medical resources, increase user comfort, reduce user stigma, and improve user cooperation. and the effectiveness of psychological assessment; real-time collection of user physiological data and behavioral data during the assessment process, using intelligent information processing technology to analyze the real-time status of users, intelligently correcting the subject’s question and answer options, and improving the accuracy and objectivity of psychological assessment results The standardization of the consultation process can solve the current situation of quantification difficulties in traditional consultation. At the same time, combined with big data technology, the user's personalized psychological characteristics are excavated and the rules are summarized to improve the accuracy of psychological assessment.
因此，将VR技术、智能传感技术、虚拟现实技术、大数据分析技术、人工智能技术与传统心理评估方法相结合，可以弥补传统方法的诸多不足，并且为评估心理状态提供了一种新的方法及思路。Therefore, the combination of VR technology, intelligent sensing technology, virtual reality technology, big data analysis technology, artificial intelligence technology and traditional psychological assessment methods can make up for many shortcomings of traditional methods and provide a new method for assessing psychological state. methods and ideas.
发明内容Contents of the invention
为解决现有技术所存在的问题，本发明提供一种基于虚拟现实技术的心理评估方法，该方法将VR技术、智能传感技术、大数据分析技术、人工智能技术与传统心理评估方法相结合，提升了心理评估的准确性，同时还有效节约了医疗资源。In order to solve the problems existing in the prior art, the present invention provides a psychological assessment method based on virtual reality technology, which combines VR technology, intelligent sensing technology, big data analysis technology, artificial intelligence technology and traditional psychological assessment methods , which improves the accuracy of psychological assessment and effectively saves medical resources.
本发明所采用的技术方案如下：一种基于虚拟现实技术的心理评估方法，包括以下步骤：The technical scheme adopted in the present invention is as follows: a psychological assessment method based on virtual reality technology, comprising the following steps:
S1：选定心理量表；S1: selected psychological scale;
S2：将所选定的心理量表虚拟场景化，呈现基于特定心理量表设计的评估内容；S2: Virtualize the selected psychological scale into a virtual scene, and present the evaluation content based on the design of the specific psychological scale;
S3：实时采集受试者问答数据、行为数据和生理数据；S3: Real-time collection of subject question and answer data, behavioral data and physiological data;
S4：基于受试者的问答选项，完成心理量表问题的智能跳转；S4: Based on the subject's question and answer options, complete the intelligent jump of the psychological scale questions;
S5：对量表内容进行综合智能分析，综合智能分析将行为数据通过卷积神经网络训练、问答数据通过循环神经网络训练、生理数据通过卷积神经网络训练后的输出经特征融合后输入到一个softmax层，得出心理评估模型；S5: Carry out comprehensive intelligent analysis on the content of the scale. The comprehensive intelligent analysis will input behavioral data through convolutional neural network training, question and answer data through recurrent neural network training, and physiological data through convolutional neural network training. Softmax layer, get the psychological evaluation model;
S6：采用心理评估模型对受试者初步学习获得的心理评估结果与医生标签进行比对，通过计算损失函数和梯度反向传导，对受试者的问答选项进行智能校正；S6: Use the psychological assessment model to compare the psychological assessment results obtained by the subject's preliminary learning with the doctor's label, and intelligently correct the subject's question and answer options by calculating the loss function and gradient reverse conduction;
S7：再将生理数据、行为数据和经校正的问答选项由心理评估模型计算，从而得到最终综合评估结果。S7: Calculate the physiological data, behavioral data and corrected question and answer options by the psychological evaluation model, so as to obtain the final comprehensive evaluation result.
优选地，步骤S4所述心理量表问题的智能跳转，通过设计多种关联条件判断并根据实时情况选择跳转策略；所述跳转策略包括根据医生临床经验量表中相关问题跳转、依据当前答题情况跳转和依据系统实时分析的使用者状态进行跳转。Preferably, the intelligent jump of the psychological scale problem described in step S4 is judged by designing multiple association conditions and selecting a jump strategy according to real-time conditions; the jump strategy includes jumping according to relevant questions in the doctor's clinical experience scale, Jump according to the current answering situation and jump according to the user status analyzed in real time by the system.
优选地，步骤S5对行为数据进行卷积神经网络的训练过程包括以下步骤：Preferably, step S5 carries out the training process of convolutional neural network to behavioral data and comprises the following steps:
(1)对视频采集的行为数据分别提取形状信息和光流信息；(1) Extract shape information and optical flow information from the behavioral data collected by the video;
(2)对提取的形状信息和光流信息进行预处理，得到卷积神经网络的输入图像；(2) Preprocessing the extracted shape information and optical flow information to obtain the input image of the convolutional neural network;
(3)设置卷积神经网络的参数，将形状信息和光流信息进行预处理后的输入图像分别输入两个卷积神经网络进行训练；(3) The parameters of the convolutional neural network are set, and the input image after the preprocessing of the shape information and the optical flow information is input into two convolutional neural networks for training;
(4)将经两个卷积神经网络处理后的形状信息和光流信息的特征进行融合；(4) Fusing the features of shape information and optical flow information processed by two convolutional neural networks;
(5)经步骤(4)特征融合后的输出依次输入到后续卷积层、池化层和全连接层。(5) The output after feature fusion in step (4) is sequentially input to the subsequent convolutional layer, pooling layer and fully connected layer.
优选地，步骤S5对问答数据在循环神经网络处理的步骤如下：Preferably, step S5 is as follows to the steps of question answering data processing in recurrent neural network:
(1)对语音问答数据进行自然语言处理，将语音数据转化为文本数据；(1) Perform natural language processing on voice question-and-answer data, and convert voice data into text data;
(2)对文本数据进行编码；(2) Encode the text data;
(3)将第一次问答数据输入到循环神经网络；(3) Input the question-answering data for the first time into the recurrent neural network;
(4)将本次问答数据与上一步输出结果输入到循环神经网络；(4) Input the question and answer data of this time and the output result of the previous step into the recurrent neural network;
(5)循环步骤(4)，直至问答结束。(5) Step (4) is looped until the question and answer ends.
优选地，步骤S5对生理数据进行卷积神经网络的训练过程包括以下步骤:Preferably, step S5 carries out the training process of convolutional neural network to physiological data and comprises the following steps:
(21)对生理数据进行短时傅里叶变换，获取多通道频谱图；(21) Perform short-time Fourier transform on the physiological data to obtain a multi-channel spectrogram;
(22)将多通道频谱图采用卷积神经网络进行训练。(22) The multi-channel spectrogram is trained with a convolutional neural network.
从以上技术方案可知，本发明基于虚拟现实技术，融合被评测对象心理量表测定、行为数据和实时监测的生理数据，通过基于带有医生标记的数据经由卷积神经网络和循环神经网络等深度学习算法得到心理评估模型，该模型具有临床经验，可独立智能评估没有医生标记的数据。同时，本发明将受试者的心理评估结果与医生标签进行比对，通过计算损失函数和梯度反向传导对问答选项进行智能校正，校正后的问答数据能更好地反映受试者真实心理状态，再经由心理评估模型计算，从而得到最终综合评估结果。与现有技术相比，本发明至少有如下有益效果：It can be seen from the above technical solutions that the present invention is based on virtual reality technology, integrates the psychological scale measurement, behavioral data and real-time monitoring physiological data of the evaluated object, and passes through convolutional neural network and recurrent neural network based on the data with doctor's mark. The learning algorithm results in a psychological assessment model with clinical experience that independently intelligently assesses data without physician labelling. At the same time, the present invention compares the psychological evaluation results of the subjects with the doctor's labels, intelligently corrects the question and answer options by calculating the loss function and gradient reverse conduction, and the corrected question and answer data can better reflect the real psychology of the subjects The state is then calculated by the psychological evaluation model to obtain the final comprehensive evaluation result. Compared with the prior art, the present invention has at least the following beneficial effects:
本发明将心理量表评估过程具象化，利用虚拟现实技术将量表问题转化成为便于受试者理解的虚拟场景，同时将医生观测数据数字化，并采集受试者生理数据，将问答数据、生理数据和行为数据进行综合分析，获得受试者心理评估结果，同时评估结果与医生标签进行比对实现受试者问答选项的智能校正。利用传统的医生经验与客观的生理参数，可以提升心理评估的准确性，帮助用户及时了解自身心理状态，帮助用户管理心理健康，预防心理疾病的发生。同时可以减少医生工作量，节约医疗资源。而且本方法为医学研究提供了新的研究思路和方法，有利于找到相关情绪、心理判断标志物及机制。本方法采集得到大量的用户数据可以用于实现云计算，运用机器学习的方法挖掘出用户个性化心理特征，进一步提供针对性的心理辅导。The present invention concretizes the evaluation process of psychological scales, uses virtual reality technology to convert scale questions into virtual scenes that are easy for subjects to understand, and at the same time digitizes doctor observation data, collects physiological data of subjects, and converts question and answer data, physiological The data and behavioral data are comprehensively analyzed to obtain the psychological evaluation results of the subjects. At the same time, the evaluation results are compared with the doctor's labels to realize the intelligent correction of the subject's question and answer options. Using traditional doctor's experience and objective physiological parameters can improve the accuracy of psychological assessment, help users understand their own mental state in a timely manner, help users manage their mental health, and prevent the occurrence of mental illness. At the same time, it can reduce the workload of doctors and save medical resources. Moreover, this method provides new research ideas and methods for medical research, and is conducive to finding markers and mechanisms of related emotions and psychological judgments. A large amount of user data collected by this method can be used to implement cloud computing, and machine learning methods are used to mine personalized psychological characteristics of users to further provide targeted psychological counseling.
附图说明Description of drawings
图1为本发明心理评估流程图；Fig. 1 is the psychological assessment flowchart of the present invention;
图2为综合智能分析原理图；Figure 2 is a schematic diagram of comprehensive intelligent analysis;
图3为智能矫正原理图；Figure 3 is a schematic diagram of intelligent correction;
图4为行为数据处理过程图；Fig. 4 is a process diagram of behavioral data processing;
图5为问答数据处理的网络结构图；Fig. 5 is a network structure diagram of question answering data processing;
图6为生理数据处理过程图。Fig. 6 is a diagram of the physiological data processing process.
具体实施方式Detailed ways
本发明提出了一种基于虚拟现实技术的心理评估方法，将心理量表评估过程具象化，利用虚拟现实技术将量表问题转化成为便于受试者理解的虚拟场景；同时将医生观测数据数字化，采集受试者生理数据，处理受试者量表问答数据，将三者结合进行综合分析，同时对受试者无效作答智能校正。避免了受试者因为外界干扰和医生观测的不同造成评估结果的差异，以及因病耻感而回避回答隐私问题，为此沉浸式虚拟现实内容的多样性和真实性为医生评估带来很大的方便，同时在特定情境下基于大数据技术能挖掘出受试者更多的信息。The present invention proposes a psychological assessment method based on virtual reality technology, which visualizes the assessment process of psychological scales, and uses virtual reality technology to convert scale questions into virtual scenes that are easy for subjects to understand; at the same time, the doctor's observation data is digitized, Collect the physiological data of the subjects, process the question and answer data of the subjects' scales, combine the three for comprehensive analysis, and intelligently correct the invalid answers of the subjects at the same time. It avoids the differences in evaluation results caused by external interference and differences in doctor's observations, and avoids answering privacy questions due to stigma. Therefore, the diversity and authenticity of immersive virtual reality content brings great benefits to doctors' evaluation. It is convenient, and at the same time, based on big data technology in specific situations, more information about subjects can be mined.
本发明心理评估方法包括心理量表的虚拟现实呈现、语音识别及自然语言处理技术、评估问题的智能跳转逻辑、受试者问答选项的智能校正算法、综合智能分析算法，如图1，具体包括如下步骤：The psychological assessment method of the present invention includes virtual reality presentation of psychological scales, speech recognition and natural language processing technology, intelligent jump logic for assessment questions, intelligent correction algorithm for subject question and answer options, and comprehensive intelligent analysis algorithm, as shown in Figure 1, specifically Including the following steps:
步骤S1：选定心理量表；Step S1: Select the psychological scale;
步骤S2：将所选定的心理量表虚拟场景化，呈现基于特定心理量表设计的评估内容；Step S2: Virtualize the selected psychological scale into a virtual scene, and present the evaluation content based on the design of the specific psychological scale;
所述心理量表的虚拟现实呈现，利用虚拟现实技术，将心理量表转化为虚拟空间中虚拟医生的提问形式。心理量表虚拟场景化，即通过对情绪空间进行分解，制作具有特定情绪诱发的虚拟现实场景，将心理量表问题具象化。The virtual reality presentation of the psychological scale uses virtual reality technology to transform the psychological scale into a form of questioning by a virtual doctor in a virtual space. The virtual scene of the psychological scale is to decompose the emotional space and create a virtual reality scene with specific emotional triggers to concretize the problems of the psychological scale.
将心理量表测定过程虚拟现实场景化，用于提高受试者对心理量表问题理解的准确性，提高受试者答案的有效性，从而心理评估结果更为科学有效；同时辅助于客观生理数据与行为数据，为心理状态评估提供有效又便捷的手段。其中虚拟现实场景包括但不限于虚拟医生形象设计、虚拟空间设计、虚拟医生问询形式。The virtual reality scene of the psychological scale measurement process is used to improve the accuracy of the subjects' understanding of the psychological scale questions and the validity of the subjects' answers, so that the psychological assessment results are more scientific and effective; at the same time, it assists the objective physiological Data and behavioral data provide an effective and convenient means for mental state assessment. The virtual reality scene includes but is not limited to virtual doctor image design, virtual space design, and virtual doctor inquiry form.
步骤S3：采集受试者实时问答数据、实时行为数据和实时生理数据；Step S3: collecting real-time question and answer data, real-time behavioral data and real-time physiological data of subjects;
所述采集的受试者实时生理数据，包括但不限于脑电、脑血流、脉搏、心电、肌电、体温、皮电、血氧浓度。用于分析受试者问答过程中的心理状态，作为心理评估模型的输入以及用于智能跳转逻辑的条件判断。The collected real-time physiological data of the subject include but not limited to EEG, cerebral blood flow, pulse, ECG, EMG, body temperature, GG, and blood oxygen concentration. It is used to analyze the psychological state of the subject during the question-and-answer process, as the input of the psychological evaluation model and the conditional judgment for the intelligent jump logic.
步骤S4：基于受试者的问答选项，完成心理量表问题的智能跳转；Step S4: Based on the subject's question and answer options, complete the intelligent jump of the psychological scale questions;
所述心理量表问题的智能跳转，通过设计多种关联条件判断并根据实时情况选择跳转策略，包括但不仅限于根据医生临床经验量表中相关问题跳转、依据当前答题情况跳转、依据系统实时分析的使用者状态进行跳转。The intelligent jumping of the psychological scale questions is judged by designing a variety of associated conditions and selecting a jumping strategy according to real-time conditions, including but not limited to jumping according to relevant questions in the doctor's clinical experience scale, jumping according to the current answering situation, Jump according to the user status analyzed in real time by the system.
步骤S5：对量表内容进行综合智能分析；Step S5: Carry out comprehensive intelligent analysis on the contents of the scale;
综合智能分析方法，用于分析所采集的用户数据，基于长时间累积的大量用户数据进行个性化心理特征挖掘与规律总结。基于带有医生标记的数据经由卷积神经网络和循环神经网络等深度学习算法得到心理评估模型。同时，本方法对受试者初步学习获得的心理评估结果与医生标签进行比对，通过计算损失函数和梯度反向传导，从而实现受试者量表作答的智能校正。将经校正的量表作答与行为和生理数据再次输入心理评估模型，得出最终心理评估报告。The comprehensive intelligent analysis method is used to analyze the collected user data, based on a large amount of user data accumulated over a long period of time to carry out personalized psychological feature mining and rule summary. Based on the data marked by doctors, the psychological assessment model is obtained through deep learning algorithms such as convolutional neural network and recurrent neural network. At the same time, this method compares the psychological assessment results obtained by the subject's preliminary learning with the doctor's label, and realizes the intelligent correction of the subject's scale answer by calculating the loss function and gradient reverse conduction. The corrected scale responses and behavioral and physiological data were re-input into the psychological assessment model to obtain the final psychological assessment report.
具体来说，综合分析首先对行为数据、生理数据、问答数据分别进行卷积神经网络、循环神经网络、卷积神经网络的训练，将三类数据训练后的输出进行特征融合，输入到Softmax层。Specifically, the comprehensive analysis first conducts convolutional neural network, recurrent neural network, and convolutional neural network training on behavioral data, physiological data, and question-and-answer data, and then performs feature fusion on the outputs of the three types of data after training, and inputs them into the Softmax layer. .
对实时行为数据进行处理，如图4所示，输入视频材料(受试者的行为信息)，提取其中形状信息和光流信息，两者分别输入到一个卷积神经网络中，两个卷积神经网络结构相同，包括第一卷积层、第一池化层、第二卷积层、第二池化层、第三卷积层1、第三卷积层2、第三池化层，两个卷积神经网络进行特征融合后，输入到第四卷积层、第四池化层、第五全连接层、第六全连接层。所述第一卷积层、第一池化层、第二卷积层、第二池化层、第三卷积层1、第三卷积层2、第三池化层依次相连，所述卷积神经网络、第四卷积层、第四池化层、第五全连接层、第六全连接层依次相连。Process the real-time behavioral data, as shown in Figure 4, input the video material (behavior information of the subject), extract the shape information and optical flow information, and input the two into a convolutional neural network respectively, and two convolutional neural networks The network structure is the same, including the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the third convolutional layer 1, the third convolutional layer 2, and the third pooling layer. After the feature fusion of the first convolutional neural network, it is input to the fourth convolutional layer, the fourth pooling layer, the fifth fully connected layer, and the sixth fully connected layer. The first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the third convolutional layer 1, the third convolutional layer 2, and the third pooling layer are connected in sequence, and the The convolutional neural network, the fourth convolutional layer, the fourth pooling layer, the fifth fully connected layer, and the sixth fully connected layer are connected in sequence.
对实时行为数据进行卷积神经网络的训练过程包括以下步骤：The training process of a convolutional neural network on real-time behavioral data includes the following steps:
(1)对视频采集的行为数据分别提取形状信息和光流信息；(1) Extract shape information and optical flow information from the behavioral data collected by the video;
(2)对提取的形状信息和光流信息进行预处理，得到卷积神经网络的输入图像；(2) Preprocessing the extracted shape information and optical flow information to obtain the input image of the convolutional neural network;
(3)设置卷积神经网络的参数，将进行预处理后的输入图像输入卷积神经网络进行训练。卷积神经网络对输入图像的训练，具体步骤如下：(3) Set the parameters of the convolutional neural network, and input the preprocessed input image into the convolutional neural network for training. The training of the convolutional neural network on the input image, the specific steps are as follows:
(31)输入图像输入第一卷积层，对其进行大小为3*3，步长为1，填充距离为1的卷积操作，一共使用3个卷积核；(31) The input image is input to the first convolutional layer, and a convolution operation with a size of 3*3, a step size of 1, and a padding distance of 1 is performed on it, and a total of 3 convolution kernels are used;
(32)经第一卷积层输出的图片输入到第一池化层，进行最大池操作，池化块的大小为2*2，步长为2；(32) The picture output by the first convolutional layer is input to the first pooling layer, and the maximum pooling operation is performed. The size of the pooling block is 2*2, and the step size is 2;
(33)经第一池化层输出的图片输入到第二卷积层，对其进行大小为3*3，步长为1，填充距离为1的卷积操作，一共使用3个卷积核；(33) The picture output by the first pooling layer is input to the second convolutional layer, and the convolution operation is performed on it with a size of 3*3, a step size of 1, and a padding distance of 1, using a total of 3 convolution kernels ;
(34)经第二卷积层输出的图片输入到第二池化层，进行最大池操作，池化块的大小为2*2，步长为2；(34) The picture output by the second convolutional layer is input to the second pooling layer, and the maximum pooling operation is performed. The size of the pooling block is 2*2, and the step size is 2;
(35)经第二池化层输出的图片输入到第三卷积层1，对其进行大小为3*3，步长为1，填充距离为1的卷积操作，一共使用3个卷积核；(35) The picture output by the second pooling layer is input to the third convolutional layer 1, and the convolution operation is performed on it with a size of 3*3, a step size of 1, and a padding distance of 1. A total of 3 convolutions are used nuclear;
(36)经第三卷积层1输出的图片输入到第三卷积层2，对其进行大小为3*3，步长为1，填充距离为1的卷积操作，一共使用3个卷积核；(36) The picture output by the third convolutional layer 1 is input to the third convolutional layer 2, and the convolution operation with a size of 3*3, a step size of 1, and a padding distance of 1 is performed on it, and a total of 3 volumes are used Accumulation;
(37)经第三卷积层2输出的图片输入到第三池化层，进行最大池操作，池化块的大小为2*2，步长为2.(37) The picture output by the third convolutional layer 2 is input to the third pooling layer, and the maximum pooling operation is performed. The size of the pooling block is 2*2, and the step size is 2.
(4)将经两个卷积神经网络处理后的形状信息和光流信息的特征进行融合；(4) Fusing the features of shape information and optical flow information processed by two convolutional neural networks;
(5)经步骤(4)特征融合后的输出输入到第四卷积层,对其进行大小为3*3，步长为1，填充距离为1的卷积操作，一共使用3个卷积核；(5) The output after feature fusion in step (4) is input to the fourth convolutional layer, and a convolution operation with a size of 3*3, a step size of 1, and a padding distance of 1 is performed on it, and a total of 3 convolutions are used nuclear;
(6)经第四卷积层输出的图片输入到第四池化层，进行最大池操作，池化块的大小为2*2，步长为2；(6) The picture output by the fourth convolutional layer is input to the fourth pooling layer, and the maximum pooling operation is performed. The size of the pooling block is 2*2, and the step size is 2;
(7)经第四池化层输出的图片输入到第五全连接层；(7) The picture output by the fourth pooling layer is input to the fifth fully connected layer;
(8)经第五全连接层输出的图片输入到第六全连接层。(8) The picture output by the fifth fully connected layer is input to the sixth fully connected layer.
对问答数据进行处理的网络结构如图5所示，是一个长短期记忆循环神经网络。所述问答数据在长短期记忆循环神经网络处理的步骤如下：The network structure for processing question-answer data is shown in Figure 5, which is a long-short-term memory recurrent neural network. The steps for processing the question-and-answer data in the long-short-term memory recurrent neural network are as follows:
(1)对语音问答数据进行自然语言处理，将语音数据转化为文本数据；(1) Perform natural language processing on voice question-and-answer data, and convert voice data into text data;
基于智能语音识别与自然语言处理算法，将受试者回答问题的语音信号转化为文本，结合医学词库经行自然语言处理得到实时答题结果，其中包括但不仅限于分词算法、关键词提取、文本情绪识别算法。Based on intelligent speech recognition and natural language processing algorithms, the voice signals of subjects answering questions are converted into texts, and real-time answer results are obtained through natural language processing combined with medical thesaurus, including but not limited to word segmentation algorithms, keyword extraction, text Emotion recognition algorithm.
(2)对文本数据进行编码；(2) Encode the text data;
(3)将第一次问答数据(即问答数据1)输入到长短期记忆循环神经网络；(3) Input the first question-and-answer data (i.e. question-and-answer data 1) into the long-short-term memory recurrent neural network;
(4)将本次问答数据与上一步输出结果输入到长短期记忆循环神经网络；(4) Input the question and answer data of this time and the output result of the previous step into the long short-term memory recurrent neural network;
(5)循环步骤(4)，直至问答结束。(5) Step (4) is looped until the question and answer ends.
采用卷积神经网络对生理数据进行处理的过程如图6所示，输入实时监测的生理数据，经过短时傅里叶变换，获取生理数据的多通道频谱图，输入到一个卷积神经网络，该卷积神经网络包括第一卷积层、第二卷积层、第三池化层及第四全连接层。所述生理数据卷积神经网络的训练过程包括以下步骤:The process of using convolutional neural network to process physiological data is shown in Figure 6. The physiological data monitored in real time is input, and after short-time Fourier transform, the multi-channel spectrogram of physiological data is obtained and input to a convolutional neural network. The convolutional neural network includes a first convolutional layer, a second convolutional layer, a third pooling layer and a fourth fully connected layer. The training process of described physiological data convolutional neural network comprises the following steps:
(1)对生理数据进行短时傅里叶变换，获取多通道频谱图；(1) Perform short-time Fourier transform on the physiological data to obtain a multi-channel spectrogram;
(2)设置所述卷积神经网络的参数，将多通道频谱图输入所述卷积神经网络进行训练。生理数据在卷积神经网络的训练步骤如下：(2) Setting the parameters of the convolutional neural network, and inputting the multi-channel spectrogram into the convolutional neural network for training. The training steps of physiological data in convolutional neural network are as follows:
(21)对生理数据进行短时傅里叶变换，获取多通道频谱图；(21) Perform short-time Fourier transform on the physiological data to obtain a multi-channel spectrogram;
(22)将频谱图输入第一卷积层，对其进行大小为15*3，步长为1，填充距离为1的卷积操作，一共使用2个卷积核；(22) Input the spectrogram into the first convolutional layer, perform a convolution operation with a size of 15*3, a step size of 1, and a padding distance of 1, using a total of 2 convolution kernels;
(23)将经第一卷积层输出的图输入到第二卷积层，对其进行大小为1*1，步长为1，填充距离为1的卷积操作，一共使用1个卷积核；(23) Input the image output by the first convolutional layer to the second convolutional layer, perform a convolution operation with a size of 1*1, a step size of 1, and a padding distance of 1, using a total of 1 convolution nuclear;
(24)将经第二卷积层输出的图输入到第三池化层，进行最大池操作，池化块的大小为2*2，步长为2；(24) Input the image output by the second convolutional layer to the third pooling layer, and perform the maximum pooling operation, the size of the pooling block is 2*2, and the step size is 2;
(25)经第三池化层输出的图片输入到第四全连接层。(25) The picture output by the third pooling layer is input to the fourth fully connected layer.
行为数据的卷积神经网络、问答数据的循环神经网络、生理数据的卷积神经网络的输出经特征融合后输入到一个softmax层，得出心理评估模型。所述特征融合将行为数据的卷积神经网络、问答数据的循环神经网络、生理数据的卷积神经网络的输出结果全串联。The output of the convolutional neural network for behavioral data, the recurrent neural network for question-and-answer data, and the convolutional neural network for physiological data is input into a softmax layer after feature fusion to obtain a psychological evaluation model. The feature fusion connects the output results of the convolutional neural network for behavioral data, the recurrent neural network for question-answering data, and the convolutional neural network for physiological data in series.
步骤S6：智能校正错误问答选项；Step S6: intelligently correcting wrong question and answer options;
本步骤对受试者的错误问答选项进行智能校正，通过将受试者问答数据、行为数据与生理数据进行训练，输出心理评估结果与医生标签进行比对，计算损失函数和梯度反向传导对问答选项，校正受试者的错误回答，如图3所示。This step intelligently corrects the wrong question and answer options of the subjects. By training the subject’s question and answer data, behavioral data and physiological data, the output psychological assessment results are compared with the doctor’s labels, and the loss function and gradient reverse conduction pair are calculated. The question-and-answer option corrects the wrong answers of the subjects, as shown in Figure 3.
如图2、3，本发明在步骤S5对量表内容进行综合智能分析以及步骤S6智能校正错误问答选项时，均对受试者行为数据进行特征融合处理，模拟医生临床评估过程观察受试者状态，记录受试者实时行为数据，包括但不仅限于表情、声音、眼动、肢体动作。利用表情识别、声音及其情绪识别、动作识别等技术，用于综合智能分析算法的输入以及用于智能跳转逻辑的条件判断。对受试者的实时行为数据进行语音识别及自然语言处理，实时采集受试者的语音信号并转化为文本，对文本进行自然语言处理分析语义与情绪。As shown in Figures 2 and 3, when the present invention conducts comprehensive intelligent analysis of the scale content in step S5 and intelligently corrects wrong question and answer options in step S6, it performs feature fusion processing on the subject's behavior data, simulating the doctor's clinical evaluation process to observe the subject Status, recording real-time behavioral data of subjects, including but not limited to expressions, voices, eye movements, and body movements. Using facial expression recognition, voice and emotion recognition, action recognition and other technologies, it is used for the input of comprehensive intelligent analysis algorithm and the conditional judgment for intelligent jump logic. Perform speech recognition and natural language processing on the real-time behavior data of the subjects, collect the speech signals of the subjects in real time and convert them into text, and perform natural language processing on the text to analyze semantics and emotions.
步骤S7：得到最终受试者心理评估报告。Step S7: Obtain the final subject's psychological assessment report.
本发明通过综合智能分析，将带有医生标定的数据经由卷积神经网络和循环神经网络等深度学习算法进行学习，得到心理评估模型。同时，本心理评估模型对受试者初步学习获得的心理评估结果与医生标签进行比对，通过计算损失函数和梯度反向传导，对问答选项进行智能校正。再将生理数据、行为数据和经校正的问答选项由心理评估模型计算，从而得到最终综合评估结果。In the present invention, through comprehensive intelligent analysis, the data with doctor's calibration is learned through deep learning algorithms such as convolutional neural network and cyclic neural network to obtain a psychological evaluation model. At the same time, this psychological assessment model compares the psychological assessment results obtained by the subjects' preliminary learning with the doctor's label, and intelligently corrects the question and answer options by calculating the loss function and gradient reverse conduction. Then the physiological data, behavioral data and corrected question and answer options are calculated by the psychological evaluation model to obtain the final comprehensive evaluation result.
本发明模拟现实心理评估过程，通过心理量表问答智能逻辑，实现虚拟现实场景中的虚拟医生与受试者智能人机交互，实时观察与记录受试者行为数据、生理数据、问答数据等；基于带有医生标记的数据经由卷积神经网络和循环神经网络等深度学习算法得到心理评估模型，该模型具有临床经验，可独立智能评估没有医生标记的数据。The present invention simulates the realistic psychological evaluation process, realizes the intelligent human-computer interaction between the virtual doctor and the subject in the virtual reality scene, observes and records the subject's behavior data, physiological data, question and answer data, etc. in real time through the intelligent logic of psychological scale question and answer; Based on the data marked by doctors, the psychological assessment model is obtained through deep learning algorithms such as convolutional neural network and recurrent neural network. This model has clinical experience and can independently and intelligently evaluate data without doctor marks.
如上所述，便可较好地实现本发明。As described above, the present invention can be preferably carried out.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710839186.3ACN107799165A (en) | 2017-09-18 | 2017-09-18 | A kind of psychological assessment method based on virtual reality technology |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710839186.3ACN107799165A (en) | 2017-09-18 | 2017-09-18 | A kind of psychological assessment method based on virtual reality technology |
| Publication Number | Publication Date |
|---|---|
| CN107799165Atrue CN107799165A (en) | 2018-03-13 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710839186.3APendingCN107799165A (en) | 2017-09-18 | 2017-09-18 | A kind of psychological assessment method based on virtual reality technology |
| Country | Link |
|---|---|
| CN (1) | CN107799165A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345873A (en)* | 2018-03-22 | 2018-07-31 | 哈尔滨工业大学 | A kind of multiple degrees of freedom body motion information analytic method based on multilayer convolutional neural networks |
| CN108428475A (en)* | 2018-05-15 | 2018-08-21 | 段新 | Biofeedback training system based on human body physiological data monitoring and virtual reality |
| CN108461126A (en)* | 2018-03-19 | 2018-08-28 | 傅笑 | In conjunction with virtual reality（VR）The novel intelligent psychological assessment of technology and interfering system |
| CN108453736A (en)* | 2018-03-22 | 2018-08-28 | 哈尔滨工业大学 | A kind of multiple degrees of freedom synchronization myoelectricity control method based on deep learning |
| CN108665962A (en)* | 2018-04-17 | 2018-10-16 | 上海大学 | A digital neuropsychological test system and method |
| CN108805089A (en)* | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Based on multi-modal Emotion identification method |
| CN108877941A (en)* | 2018-06-07 | 2018-11-23 | 衡阳师范学院 | The Evaluation on psychological health dual system of teenager's rebel's phase |
| CN108888277A (en)* | 2018-04-26 | 2018-11-27 | 深圳市科思创动科技有限公司 | Psychological test method, system and terminal device |
| CN108937967A (en)* | 2018-05-29 | 2018-12-07 | 智众伟业（天津）科技有限公司南宁分公司 | A kind of psychology data memory promotion detection method and system based on VR technology |
| CN108983966A (en)* | 2018-06-13 | 2018-12-11 | 济南博图信息技术有限公司 | Reformation of convicts assessment system and method based on virtual reality and eye movement technique |
| CN108986888A (en)* | 2018-07-16 | 2018-12-11 | 上海赞彤医疗科技有限公司 | Examination anxiety and pressure regulating system and method, storage medium, operating system |
| CN109086837A (en)* | 2018-10-24 | 2018-12-25 | 高嵩 | User property classification method, storage medium, device and electronic equipment based on convolutional neural networks |
| CN109102862A (en)* | 2018-07-16 | 2018-12-28 | 上海赞彤医疗科技有限公司 | Concentrate the mind on breathing depressurized system and method, storage medium, operating system |
| CN109164910A (en)* | 2018-07-05 | 2019-01-08 | 北京航空航天大学合肥创新研究院 | For the multiple signals neural network architecture design method of electroencephalogram |
| CN109171773A (en)* | 2018-09-30 | 2019-01-11 | 合肥工业大学 | Sentiment analysis method and system based on multi-channel data |
| CN109192310A (en)* | 2018-07-25 | 2019-01-11 | 同济大学 | A kind of undergraduate psychological behavior unusual fluctuation scheme Design method based on big data |
| CN109171774A (en)* | 2018-09-30 | 2019-01-11 | 合肥工业大学 | Personality analysis method and system based on multi-channel data |
| CN109242014A (en)* | 2018-08-29 | 2019-01-18 | 沈阳康泰电子科技股份有限公司 | A kind of deep neural network psychology semanteme marking method based on the micro- feature of multi-source |
| CN109325457A (en)* | 2018-09-30 | 2019-02-12 | 合肥工业大学 | Sentiment analysis method and system based on multi-channel data and recurrent neural network |
| CN109344909A (en)* | 2018-10-30 | 2019-02-15 | 咪付（广西）网络技术有限公司 | A kind of personal identification method based on multichannel convolutive neural network |
| CN109480824A (en)* | 2018-12-11 | 2019-03-19 | 武汉中旗生物医疗电子有限公司 | Processing method, device and the server of EGC waveform data |
| CN109493885A (en)* | 2018-11-13 | 2019-03-19 | 平安科技（深圳）有限公司 | Psychological condition assessment and adjusting method, device and storage medium, server |
| CN109711383A (en)* | 2019-01-07 | 2019-05-03 | 重庆邮电大学 | Time-frequency domain-based convolutional neural network motor imagery EEG signal recognition method |
| CN109960723A (en)* | 2019-04-12 | 2019-07-02 | 浙江连信科技有限公司 | A kind of interactive system and method for psychological robot |
| CN109994166A (en)* | 2019-04-02 | 2019-07-09 | 上海市精神卫生中心（上海市心理咨询培训中心） | The assessment of teenager's game obstacle, psychological characteristics investigation and Psychological Intervention System |
| CN109998570A (en)* | 2019-03-11 | 2019-07-12 | 山东大学 | Inmate's psychological condition appraisal procedure, terminal, equipment and system |
| CN110060762A (en)* | 2019-04-22 | 2019-07-26 | 北京师范大学 | A kind of mental development level appraisal procedure and system based on multiple-factor scale data |
| CN110096145A (en)* | 2019-04-11 | 2019-08-06 | 湖北大学 | Psychological condition display methods and device based on mixed reality and neural network |
| CN110123266A (en)* | 2019-05-05 | 2019-08-16 | 北京航空航天大学 | A kind of maneuvering decision modeling method based on multi-modal physiologic information |
| CN110288974A (en)* | 2018-03-19 | 2019-09-27 | 北京京东尚科信息技术有限公司 | Voice-based Emotion identification method and device |
| CN110298301A (en)* | 2019-04-17 | 2019-10-01 | 国网江苏省电力有限公司 | A kind of establishment officer's phychology prediction technique |
| CN110522983A (en)* | 2018-05-23 | 2019-12-03 | 深圳先进技术研究院 | Artificial intelligence-based brain stimulation system, method, device and storage medium |
| CN110610754A (en)* | 2019-08-16 | 2019-12-24 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | An immersive wearable diagnostic and therapeutic device |
| CN110916691A (en)* | 2019-12-11 | 2020-03-27 | 陕西学前师范学院 | University student's psychological state testing arrangement |
| CN111028919A (en)* | 2019-12-03 | 2020-04-17 | 北方工业大学 | Phobia self-diagnosis and treatment system based on artificial intelligence algorithm |
| CN111134694A (en)* | 2019-12-20 | 2020-05-12 | 浙江连信科技有限公司 | Psychological consultation analysis method and device based on human-computer interaction |
| CN111524578A (en)* | 2020-06-19 | 2020-08-11 | 智恩陪心(北京)科技有限公司 | Psychological assessment device, method and system based on electronic psychological sand table |
| CN111724882A (en)* | 2020-06-30 | 2020-09-29 | 重庆医科大学附属第一医院 | A self-friend mental training system and method based on virtual reality technology |
| CN112086169A (en)* | 2020-09-19 | 2020-12-15 | 北京心灵力量科技有限公司 | Interactive psychological persuasion system adopting psychological data labeling modeling |
| CN112117002A (en)* | 2020-09-09 | 2020-12-22 | 温州大学 | A new type of intelligent psychological assessment intervention system and method combined with virtual reality technology |
| CN112185558A (en)* | 2020-09-22 | 2021-01-05 | 珠海中科先进技术研究院有限公司 | Mental health and rehabilitation evaluation method, device and medium based on deep learning |
| CN112185517A (en)* | 2020-10-20 | 2021-01-05 | 浙江连信科技有限公司 | Information processing method and device for public staff psychological construction |
| CN112182339A (en)* | 2020-11-03 | 2021-01-05 | 深圳市艾利特医疗科技有限公司 | Psychological assessment method and system |
| CN112599245A (en)* | 2020-12-16 | 2021-04-02 | 中国人民解放军总医院第八医学中心 | Mental health index evaluation method and system |
| CN112652381A (en)* | 2020-12-18 | 2021-04-13 | 中国人民解放军总医院第八医学中心 | Mental health correction plan generation method and system |
| CN112716494A (en)* | 2021-01-18 | 2021-04-30 | 上海对外经贸大学 | Mental health condition analysis algorithm based on micro-expression and brain wave analysis algorithm |
| CN113470787A (en)* | 2021-07-09 | 2021-10-01 | 福州大学 | Emotional recognition and desensitization training effect evaluation method based on neural network |
| CN113628724A (en)* | 2021-07-01 | 2021-11-09 | 江苏嘉纳宝医疗科技有限公司 | Evaluation intervention method for violent fear psychology based on virtual reality technology |
| CN113793687A (en)* | 2021-09-27 | 2021-12-14 | 盐城师范学院 | A mental health dynamic management system and method |
| CN114582501A (en)* | 2020-12-01 | 2022-06-03 | 中移(成都)信息通信科技有限公司 | Methods, apparatus, equipment and storage media for evaluating psychological assessment results |
| CN114870197A (en)* | 2022-05-20 | 2022-08-09 | 中国人民解放军空军军医大学 | Immersive psychological massage instrument |
| CN115607156A (en)* | 2022-10-21 | 2023-01-17 | 厦门诸格量科技有限公司 | Multi-mode-based psychological cognition screening evaluation method and system and storage medium |
| CN116312970A (en)* | 2023-03-23 | 2023-06-23 | 苏州复变医疗科技有限公司 | Intelligent interaction method and device for psychological assessment |
| CN116825361A (en)* | 2023-08-25 | 2023-09-29 | 佛山市龙生光启科技有限公司 | A fully automatic mental state assessment system based on facial recognition |
| CN117690582A (en)* | 2024-01-31 | 2024-03-12 | 吉林大学 | Information management system and method for nursing workstation |
| CN117854724A (en)* | 2023-12-28 | 2024-04-09 | 医养康(北京)健康管理有限公司 | Disability prediction system, disability prediction method, computer device, and storage medium |
| CN118016249A (en)* | 2024-02-19 | 2024-05-10 | 中山大学孙逸仙纪念医院 | A method and system for alleviating preoperative anxiety based on virtual reality technology |
| CN118486455A (en)* | 2024-07-16 | 2024-08-13 | 杭州虚之实科技有限公司 | A multimodal physiological data evaluation system based on virtual reality technology |
| CN118486431A (en)* | 2024-05-17 | 2024-08-13 | 肇庆医学高等专科学校 | A psychological health assessment method and system based on user behavior data |
| CN118711824A (en)* | 2024-08-30 | 2024-09-27 | 南昌大学第一附属医院 | A method for analyzing the recovery status of critically ill patients based on parallel feature extraction |
| IT202300006468A1 (en) | 2023-04-03 | 2024-10-03 | Lorenzo Maria Rosaria Di | ORGANIZATIONAL FRAMEWORK SYSTEMS AND METHODS FOR PSYCHOSOCIAL MONITORING OF LEADERSHIP AND MANAGERIAL RISK |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105893780A (en)* | 2016-05-10 | 2016-08-24 | 华南理工大学 | Mental and psychological assessment method based on VR interaction and brain wave and cerebral blood flow monitoring |
| CN105975780A (en)* | 2016-05-10 | 2016-09-28 | 华南理工大学 | Machine inquiry system based on virtual reality interaction technology |
| CN107145878A (en)* | 2017-06-01 | 2017-09-08 | 重庆邮电大学 | Abnormal Behavior Detection Method for the Elderly Based on Deep Learning |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105893780A (en)* | 2016-05-10 | 2016-08-24 | 华南理工大学 | Mental and psychological assessment method based on VR interaction and brain wave and cerebral blood flow monitoring |
| CN105975780A (en)* | 2016-05-10 | 2016-09-28 | 华南理工大学 | Machine inquiry system based on virtual reality interaction technology |
| CN107145878A (en)* | 2017-06-01 | 2017-09-08 | 重庆邮电大学 | Abnormal Behavior Detection Method for the Elderly Based on Deep Learning |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110288974B (en)* | 2018-03-19 | 2024-04-05 | 北京京东尚科信息技术有限公司 | Emotion recognition method and device based on voice |
| CN108461126A (en)* | 2018-03-19 | 2018-08-28 | 傅笑 | In conjunction with virtual reality（VR）The novel intelligent psychological assessment of technology and interfering system |
| CN110288974A (en)* | 2018-03-19 | 2019-09-27 | 北京京东尚科信息技术有限公司 | Voice-based Emotion identification method and device |
| CN108345873A (en)* | 2018-03-22 | 2018-07-31 | 哈尔滨工业大学 | A kind of multiple degrees of freedom body motion information analytic method based on multilayer convolutional neural networks |
| CN108453736A (en)* | 2018-03-22 | 2018-08-28 | 哈尔滨工业大学 | A kind of multiple degrees of freedom synchronization myoelectricity control method based on deep learning |
| CN108665962A (en)* | 2018-04-17 | 2018-10-16 | 上海大学 | A digital neuropsychological test system and method |
| CN108888277B (en)* | 2018-04-26 | 2021-05-04 | 深圳市科思创动科技有限公司 | Psychological test method, psychological test system and terminal equipment |
| CN108888277A (en)* | 2018-04-26 | 2018-11-27 | 深圳市科思创动科技有限公司 | Psychological test method, system and terminal device |
| CN108428475B (en)* | 2018-05-15 | 2023-09-15 | 段新 | Biological feedback training system based on human physiological data monitoring and virtual reality |
| CN108428475A (en)* | 2018-05-15 | 2018-08-21 | 段新 | Biofeedback training system based on human body physiological data monitoring and virtual reality |
| CN110522983A (en)* | 2018-05-23 | 2019-12-03 | 深圳先进技术研究院 | Artificial intelligence-based brain stimulation system, method, device and storage medium |
| US11577043B2 (en) | 2018-05-23 | 2023-02-14 | Shenzhen Institutes Of Advanced Technology | Brain stimulation system, method and apparatus based on artificial intelligence and storage medium |
| CN108937967A (en)* | 2018-05-29 | 2018-12-07 | 智众伟业（天津）科技有限公司南宁分公司 | A kind of psychology data memory promotion detection method and system based on VR technology |
| CN108877941A (en)* | 2018-06-07 | 2018-11-23 | 衡阳师范学院 | The Evaluation on psychological health dual system of teenager's rebel's phase |
| CN108983966A (en)* | 2018-06-13 | 2018-12-11 | 济南博图信息技术有限公司 | Reformation of convicts assessment system and method based on virtual reality and eye movement technique |
| CN108983966B (en)* | 2018-06-13 | 2021-07-02 | 济南博图信息技术有限公司 | Criminal reconstruction assessment system and method based on virtual reality and eye movement technology |
| CN108805089B (en)* | 2018-06-14 | 2021-06-29 | 南京云思创智信息科技有限公司 | Multi-modal-based emotion recognition method |
| CN108805089A (en)* | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Based on multi-modal Emotion identification method |
| CN109164910A (en)* | 2018-07-05 | 2019-01-08 | 北京航空航天大学合肥创新研究院 | For the multiple signals neural network architecture design method of electroencephalogram |
| CN109164910B (en)* | 2018-07-05 | 2021-09-21 | 北京航空航天大学合肥创新研究院 | Multi-signal neural network architecture design method for electroencephalogram |
| CN109102862A (en)* | 2018-07-16 | 2018-12-28 | 上海赞彤医疗科技有限公司 | Concentrate the mind on breathing depressurized system and method, storage medium, operating system |
| CN108986888A (en)* | 2018-07-16 | 2018-12-11 | 上海赞彤医疗科技有限公司 | Examination anxiety and pressure regulating system and method, storage medium, operating system |
| CN109192310A (en)* | 2018-07-25 | 2019-01-11 | 同济大学 | A kind of undergraduate psychological behavior unusual fluctuation scheme Design method based on big data |
| CN109242014A (en)* | 2018-08-29 | 2019-01-18 | 沈阳康泰电子科技股份有限公司 | A kind of deep neural network psychology semanteme marking method based on the micro- feature of multi-source |
| CN109242014B (en)* | 2018-08-29 | 2021-10-22 | 沈阳康泰电子科技股份有限公司 | Deep neural network psychological semantic annotation method based on multi-source micro-features |
| CN109325457A (en)* | 2018-09-30 | 2019-02-12 | 合肥工业大学 | Sentiment analysis method and system based on multi-channel data and recurrent neural network |
| CN109171773B (en)* | 2018-09-30 | 2021-05-18 | 合肥工业大学 | Emotion analysis method and system based on multi-channel data |
| CN109171774A (en)* | 2018-09-30 | 2019-01-11 | 合肥工业大学 | Personality analysis method and system based on multi-channel data |
| CN109171773A (en)* | 2018-09-30 | 2019-01-11 | 合肥工业大学 | Sentiment analysis method and system based on multi-channel data |
| CN109086837A (en)* | 2018-10-24 | 2018-12-25 | 高嵩 | User property classification method, storage medium, device and electronic equipment based on convolutional neural networks |
| CN109344909A (en)* | 2018-10-30 | 2019-02-15 | 咪付（广西）网络技术有限公司 | A kind of personal identification method based on multichannel convolutive neural network |
| CN109493885A (en)* | 2018-11-13 | 2019-03-19 | 平安科技（深圳）有限公司 | Psychological condition assessment and adjusting method, device and storage medium, server |
| CN109480824B (en)* | 2018-12-11 | 2021-10-08 | 武汉中旗生物医疗电子有限公司 | Method and device for processing electrocardio waveform data and server |
| CN109480824A (en)* | 2018-12-11 | 2019-03-19 | 武汉中旗生物医疗电子有限公司 | Processing method, device and the server of EGC waveform data |
| CN109711383A (en)* | 2019-01-07 | 2019-05-03 | 重庆邮电大学 | Time-frequency domain-based convolutional neural network motor imagery EEG signal recognition method |
| CN109998570A (en)* | 2019-03-11 | 2019-07-12 | 山东大学 | Inmate's psychological condition appraisal procedure, terminal, equipment and system |
| CN109994166A (en)* | 2019-04-02 | 2019-07-09 | 上海市精神卫生中心（上海市心理咨询培训中心） | The assessment of teenager's game obstacle, psychological characteristics investigation and Psychological Intervention System |
| CN110096145A (en)* | 2019-04-11 | 2019-08-06 | 湖北大学 | Psychological condition display methods and device based on mixed reality and neural network |
| CN109960723A (en)* | 2019-04-12 | 2019-07-02 | 浙江连信科技有限公司 | A kind of interactive system and method for psychological robot |
| CN110298301B (en)* | 2019-04-17 | 2021-09-07 | 国网江苏省电力有限公司 | Method for predicting psychological states of organization personnel |
| CN110298301A (en)* | 2019-04-17 | 2019-10-01 | 国网江苏省电力有限公司 | A kind of establishment officer's phychology prediction technique |
| CN110060762A (en)* | 2019-04-22 | 2019-07-26 | 北京师范大学 | A kind of mental development level appraisal procedure and system based on multiple-factor scale data |
| CN110123266A (en)* | 2019-05-05 | 2019-08-16 | 北京航空航天大学 | A kind of maneuvering decision modeling method based on multi-modal physiologic information |
| CN110610754A (en)* | 2019-08-16 | 2019-12-24 | 天津职业技术师范大学(中国职业培训指导教师进修中心) | An immersive wearable diagnostic and therapeutic device |
| CN111028919A (en)* | 2019-12-03 | 2020-04-17 | 北方工业大学 | Phobia self-diagnosis and treatment system based on artificial intelligence algorithm |
| CN110916691A (en)* | 2019-12-11 | 2020-03-27 | 陕西学前师范学院 | University student's psychological state testing arrangement |
| CN111134694A (en)* | 2019-12-20 | 2020-05-12 | 浙江连信科技有限公司 | Psychological consultation analysis method and device based on human-computer interaction |
| CN111524578A (en)* | 2020-06-19 | 2020-08-11 | 智恩陪心(北京)科技有限公司 | Psychological assessment device, method and system based on electronic psychological sand table |
| CN111524578B (en)* | 2020-06-19 | 2023-08-11 | 智恩陪心(北京)科技有限公司 | Psychological assessment device, method and system based on electronic psychological sand table |
| CN111724882A (en)* | 2020-06-30 | 2020-09-29 | 重庆医科大学附属第一医院 | A self-friend mental training system and method based on virtual reality technology |
| CN112117002A (en)* | 2020-09-09 | 2020-12-22 | 温州大学 | A new type of intelligent psychological assessment intervention system and method combined with virtual reality technology |
| CN112086169B (en)* | 2020-09-19 | 2024-02-09 | 北京心灵力量科技有限公司 | Interactive psychological dispersion system adopting psychological data labeling modeling |
| CN112086169A (en)* | 2020-09-19 | 2020-12-15 | 北京心灵力量科技有限公司 | Interactive psychological persuasion system adopting psychological data labeling modeling |
| CN112185558A (en)* | 2020-09-22 | 2021-01-05 | 珠海中科先进技术研究院有限公司 | Mental health and rehabilitation evaluation method, device and medium based on deep learning |
| CN112185517A (en)* | 2020-10-20 | 2021-01-05 | 浙江连信科技有限公司 | Information processing method and device for public staff psychological construction |
| CN112182339A (en)* | 2020-11-03 | 2021-01-05 | 深圳市艾利特医疗科技有限公司 | Psychological assessment method and system |
| CN114582501A (en)* | 2020-12-01 | 2022-06-03 | 中移(成都)信息通信科技有限公司 | Methods, apparatus, equipment and storage media for evaluating psychological assessment results |
| CN112599245A (en)* | 2020-12-16 | 2021-04-02 | 中国人民解放军总医院第八医学中心 | Mental health index evaluation method and system |
| CN112652381A (en)* | 2020-12-18 | 2021-04-13 | 中国人民解放军总医院第八医学中心 | Mental health correction plan generation method and system |
| CN112716494A (en)* | 2021-01-18 | 2021-04-30 | 上海对外经贸大学 | Mental health condition analysis algorithm based on micro-expression and brain wave analysis algorithm |
| CN113628724B (en)* | 2021-07-01 | 2024-03-12 | 江苏嘉纳宝医疗科技有限公司 | Assessment and intervention method for violent fear psychology based on virtual reality technology |
| CN113628724A (en)* | 2021-07-01 | 2021-11-09 | 江苏嘉纳宝医疗科技有限公司 | Evaluation intervention method for violent fear psychology based on virtual reality technology |
| CN113470787A (en)* | 2021-07-09 | 2021-10-01 | 福州大学 | Emotional recognition and desensitization training effect evaluation method based on neural network |
| CN113470787B (en)* | 2021-07-09 | 2024-01-30 | 福州大学 | Emotion recognition and desensitization training effect evaluation method based on neural network |
| CN113793687A (en)* | 2021-09-27 | 2021-12-14 | 盐城师范学院 | A mental health dynamic management system and method |
| CN114870197A (en)* | 2022-05-20 | 2022-08-09 | 中国人民解放军空军军医大学 | Immersive psychological massage instrument |
| CN115607156A (en)* | 2022-10-21 | 2023-01-17 | 厦门诸格量科技有限公司 | Multi-mode-based psychological cognition screening evaluation method and system and storage medium |
| CN116312970A (en)* | 2023-03-23 | 2023-06-23 | 苏州复变医疗科技有限公司 | Intelligent interaction method and device for psychological assessment |
| CN116312970B (en)* | 2023-03-23 | 2023-12-15 | 苏州复变医疗科技有限公司 | Intelligent interaction method and device for psychological assessment |
| IT202300006468A1 (en) | 2023-04-03 | 2024-10-03 | Lorenzo Maria Rosaria Di | ORGANIZATIONAL FRAMEWORK SYSTEMS AND METHODS FOR PSYCHOSOCIAL MONITORING OF LEADERSHIP AND MANAGERIAL RISK |
| EP4443360A1 (en) | 2023-04-03 | 2024-10-09 | Maria Rosaria Di Lorenzo | Organizational classification system and methods for the psychosocial monitoring of leadership and managerial risk |
| CN116825361B (en)* | 2023-08-25 | 2023-11-14 | 湘南学院 | Full-automatic psychological state assessment system based on facial recognition |
| CN116825361A (en)* | 2023-08-25 | 2023-09-29 | 佛山市龙生光启科技有限公司 | A fully automatic mental state assessment system based on facial recognition |
| CN117854724A (en)* | 2023-12-28 | 2024-04-09 | 医养康(北京)健康管理有限公司 | Disability prediction system, disability prediction method, computer device, and storage medium |
| CN117690582A (en)* | 2024-01-31 | 2024-03-12 | 吉林大学 | Information management system and method for nursing workstation |
| CN117690582B (en)* | 2024-01-31 | 2024-04-26 | 吉林大学 | Information management system and method for nursing workstation |
| CN118016249A (en)* | 2024-02-19 | 2024-05-10 | 中山大学孙逸仙纪念医院 | A method and system for alleviating preoperative anxiety based on virtual reality technology |
| CN118016249B (en)* | 2024-02-19 | 2024-09-10 | 中山大学孙逸仙纪念医院 | Preoperative anxiety relieving method and system based on virtual reality technology |
| CN118486431A (en)* | 2024-05-17 | 2024-08-13 | 肇庆医学高等专科学校 | A psychological health assessment method and system based on user behavior data |
| CN118486455A (en)* | 2024-07-16 | 2024-08-13 | 杭州虚之实科技有限公司 | A multimodal physiological data evaluation system based on virtual reality technology |
| CN118486455B (en)* | 2024-07-16 | 2024-11-01 | 杭州虚之实科技有限公司 | Multi-mode physiological data evaluation system based on virtual reality technology |
| CN118711824A (en)* | 2024-08-30 | 2024-09-27 | 南昌大学第一附属医院 | A method for analyzing the recovery status of critically ill patients based on parallel feature extraction |
| CN118711824B (en)* | 2024-08-30 | 2025-01-21 | 南昌大学第一附属医院 | A method for analyzing the recovery status of critically ill patients based on parallel feature extraction |
| Publication | Publication Date | Title |
|---|---|---|
| CN107799165A (en) | A kind of psychological assessment method based on virtual reality technology | |
| Kumar et al. | Hierarchical deep neural network for mental stress state detection using IoT based biomarkers | |
| Zucco et al. | Sentiment analysis and affective computing for depression monitoring | |
| CN105893780A (en) | Mental and psychological assessment method based on VR interaction and brain wave and cerebral blood flow monitoring | |
| CN113974589B (en) | Multimodal behavioral paradigm evaluation optimization system and cognitive ability evaluation method | |
| CN107451407A (en) | A kind of traditional Chinese medical science intelligent diagnosing method, system and traditional medical system | |
| CN112133407A (en) | Rapid intelligent emotion assessment analysis method based on voice and expression | |
| Tian et al. | ResNet-50 based technique for EEG image characterization due to varying environmental stimuli | |
| CN106407715A (en) | Health identification system and method for intelligent service robot | |
| CN115329818A (en) | Multi-modal fusion attention assessment method, system and storage medium based on VR | |
| CN109998570A (en) | Inmate's psychological condition appraisal procedure, terminal, equipment and system | |
| CN108814565A (en) | A kind of intelligent Chinese medicine health detection dressing table based on multi-sensor information fusion and deep learning | |
| CN106618481A (en) | Traditional Chinese medicine intelligent diagnosis expert system | |
| Othmani et al. | A model of normality inspired deep learning framework for depression relapse prediction using audiovisual data | |
| CN117064388A (en) | System for realizing mental disorder assessment analysis based on emotion recognition | |
| CN116491944A (en) | A Mental State Monitoring and Evaluation System Based on Smart Bracelet | |
| CN114299068A (en) | Intelligent decision-making system for evaluating psoriasis severity degree based on skin image | |
| CN115376695A (en) | Method, system and device for neuropsychological assessment and intervention based on augmented reality | |
| CN115429272B (en) | Psychological health state assessment method and system based on multi-mode physiological signals | |
| CN110610754A (en) | An immersive wearable diagnostic and therapeutic device | |
| CN115024725A (en) | Tumor treatment aid decision-making system integrating psychological state multi-parameter detection | |
| CN114305418A (en) | A data acquisition system and method for intelligent assessment of depression state | |
| CN114283912A (en) | Medical record filing method based on RTHD and artificial intelligence and cloud platform system | |
| CN120015351A (en) | Multimodal large model recognition and intelligent intervention methods and systems for mental disorders | |
| Hu et al. | Acrophobia quantified by EEG based on CNN incorporating Granger causality |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right | Effective date of registration:20181212 Address after:510330 Room 7,500, Haicheng West Street, Xingang East Road, Haizhu District, Guangzhou City, Guangdong Province Applicant after:Guangzhou Boguanwen Language Technology Co., Ltd. Address before:510640 No. five, 381 mountain road, Guangzhou, Guangdong, Tianhe District Applicant before:South China University of Technology | |
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right | Effective date of registration:20190925 Address after:510000 820, room 8, 8, 116 Heng Road, Dongguan, Guangzhou, Guangdong. Applicant after:Guangzhou Bo Wei Intelligent Technology Co., Ltd. Address before:510330 Room 7,500, Haicheng West Street, Xingang East Road, Haizhu District, Guangzhou City, Guangdong Province Applicant before:Guangzhou Boguanwen Language Technology Co., Ltd. | |
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | Application publication date:20180313 |