Disclosure of Invention
The embodiment of the invention provides a scene-based multi-role wounded handling virtual training system, which enables the whole virtual training system to run efficiently and stably through the configuration of each module in the system, thereby greatly improving the training effect.
The invention provides a scene-based multi-role wounded person treatment virtual training system, which comprises:
The system comprises a teacher end, a management end, a student end, an editing end and a monitoring end;
the management end is respectively in communication connection with the teacher end, the student end, the editing end and the monitoring end and is used for carrying out data management and data conversion on data of the teacher end, the student end, the editing end and the monitoring end;
the student end is used for training a student according to a training scene and obtaining training data of the student, wherein the training data comprises training data corresponding to the training scene and vital sign data of the student in the training scene;
the teacher end is used for controlling a training process corresponding to the training scene;
The editing end is used for editing the injury configuration of the injured person in the training scene;
the monitoring end is used for monitoring the training process of the trainee in the training scene.
In one possible design, the management end includes:
The management module is used for managing and configuring users, equipment, scenes, points, treatments, schemes, resources and systems;
the conversion module is used for converting training data of the teacher end, the student end and the monitoring end;
And the recording module is used for marking and recording the training data.
In one possible design, the management module includes:
The user management sub-module is used for managing all users logging in the system, and the users comprise students and teachers;
The device management sub-module is used for managing all hardware devices accessed to the system;
the scene management sub-module is used for managing all training scenes configured in the system;
The integral management sub-module is used for managing the integral of the student logged in by the student end in the training scene;
the treatment management sub-module is used for carrying out full-flow management on the training scheme corresponding to the training scene;
the scheme management sub-module is used for carrying out configuration management on the training scheme corresponding to the training scene;
The resource management sub-module is used for carrying out configuration management on training resources in a training scheme corresponding to the training scene;
and the system management sub-module is used for carrying out configuration management on the log and the authority of the system.
In one possible design, the disposition management submodule is specifically configured to:
Configuring a corresponding AI intelligent agent for each of a plurality of treatment processes corresponding to the training scene, wherein the plurality of treatment processes comprise using instruments, using consumables, using medicines, checking parts, executing actions, playing voices, playing sound effects, playing animations, voice recognition, starting processes, evaluating operations, equipment values, chest compressions, consciousness judgment and opening air passages;
Controlling a target AI (advanced technology attachment) agent to execute a target treatment process and acquiring treatment data corresponding to the target treatment process, wherein the target AI agent is an agent corresponding to the target treatment process in the AI agents, and the target treatment process is any one treatment process in the AI agents;
and executing corresponding operation according to the treatment data.
In one possible design, the conversion module is specifically configured to:
receiving target instructions and/or transmission data of the teacher side, the student side or the monitoring side;
Determining a target end corresponding to the target instruction and/or the transmission data, wherein the target end is the teacher end, the student end or the monitoring end;
And sending the target instruction to the target end, and/or sending the transmission data to the target end after conversion.
In one possible design, the learner terminal is specifically configured to:
downloading training resources from the management end according to the training scheme, wherein the training resources comprise scene resources, role resources and task resources;
Initializing a training scene corresponding to the training scheme;
distributing simulated task roles for the student ends based on the training scenes corresponding to the initialized training schemes, wherein the task roles comprise a doctor of a primary doctor, an anesthesiologist, an aid for surgery, an instrument nurse and a round nurse;
And monitoring vital sign data of the task roles in real time, and sending the vital sign data to the management end.
In one possible design, the management end is further configured to:
determining vital sign data of a task role corresponding to a target training scheme;
And training and judging the psychological adaptability and the compressive capacity of the task roles based on the vital sign data and the target training scheme.
In one possible design, the teacher end is specifically configured to:
newly building a training scene;
configuring training students and training schemes corresponding to the training scenes;
pairing the training student equipment corresponding to the training scene;
Checking the training progress corresponding to the training scene;
and watching the training visual angle of the training student corresponding to the training scene, and controlling whether the training scene is retrained.
In one possible design, the editing end is specifically configured to:
Editing the training scene;
editing the number of wounded persons in the training scene, the injury, posture, dressing and consumable material corresponding to wound treatment of each wounded person and the state corresponding to the wound treatment.
In one possible design, the monitoring end is specifically configured to:
Monitoring a picture corresponding to the student end and vital sign data of the student corresponding to the student end;
And storing the monitored video.
In summary, it can be seen that in the embodiment of the present invention, by the cooperative work of the teacher side, the management side, the learner side, the editing side, and the monitoring side, efficient management and real-time monitoring of the virtual training system are realized. The teacher end is responsible for creating and managing training scenes, the management end is responsible for managing and converting data, the learner end is responsible for executing training processes and collecting data, the editing end is responsible for editing training scenes and wounded information, and the monitoring end is responsible for monitoring states of the learner and recording and storing the information. The functional modules are tightly matched, so that the whole virtual training system can run efficiently and stably, and training effect and student experience are greatly improved.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
In the description that follows, specific embodiments of the invention will be described with reference to steps and symbols performed by one or more computers, unless otherwise indicated. Thus, these steps and operations will be referred to in several instances as being performed by a computer, which as referred to herein performs operations that include processing units by the computer that represent electronic signals that represent data in a structured form. This operation transforms the data or maintains it in place in the computer's memory system, which may reconfigure or otherwise alter the computer's operation in a manner well known to those skilled in the art. The data structure maintained by the data is the physical location of the memory, which has specific characteristics defined by the data format. However, the principles of the present invention are described in the foregoing text and are not meant to be limiting, and one skilled in the art will recognize that various steps and operations described below may also be implemented in hardware.
The principles of the present invention operate using many other general purpose or special purpose operations, communication environments, or configurations. Examples of computing systems, environments, and configurations that may be suitable for use with the invention include, but are not limited to, hand-held telephones, personal computers, servers, multiprocessor systems, microcomputer-based systems, mainframe computers, and distributed computing environments that include any of the above systems or devices.
The terms "first," "second," and "third," etc. in this disclosure are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion.
Virtual training systems play an increasingly important role in modern education and skill training, particularly in the fields of medical first aid, military training, and the like. By simulating a real scene, a learner can perform practical operations in a safe environment, thereby improving the ability to cope with complex situations. In recent years, with the development of virtual reality technology, a multi-role interactive virtual training system is gradually raised, and a new approach is provided for the cultivation of team cooperation and emergency treatment capability. The system can simulate a vivid training environment, can record operation data of students, provides basis for subsequent evaluation, and greatly improves training effect.
The virtual training system provided by the invention comprises the functions of simulation training based on role division, data management and analysis and remote monitoring. The simulation training based on role division generally enables students to play different roles in the virtual environment through preset scenes and task distribution, and the designated tasks are completed. The data management and analysis mainly relies on a central server to collect, sort and analyze the operation data of the students to evaluate the performances of the students. In addition, the remote monitoring function allows teachers to observe the operation process of students in real time, and smooth training is ensured.
Referring to fig. 1, fig. 1 is a system architecture diagram of a scene-based multi-role wounded handling virtual training system according to an embodiment of the present invention, the system includes:
teacher end 101, management end 102, student end 103, editing end 104 and monitoring end 105;
The management end 102 is respectively in communication connection with the teacher end 101, the student end 103, the editing end 104 and the monitoring end 105, and is configured to perform data management and data conversion on the teacher end 101, the student end 103, the editing end 104 and the monitoring end 105;
The learner terminal 103 is configured to perform corresponding training on a learner according to a training scene, and obtain training data, where the training data includes training data corresponding to the training scene and vital sign data of the learner in the training scene;
the teacher end 101 is used for controlling a training process corresponding to the training scene;
the editing end 104 is configured to edit the injury configuration of the injured person in the training scene;
the monitoring end 105 is configured to monitor a training process of the learner in the training scenario.
The management end 102 includes a management module, where the management module is configured to manage and configure users, devices, scenes, points, treatments, schemes, resources and systems;
the conversion module is used for converting training data of the teacher end, the student end and the monitoring end;
And the recording module is used for marking and recording the training data.
The management end 102 is a core of the whole system and generally operates on a server host, and functions of the management end 102 include user management, device management, scene management, point management, disposition management, scheme management, resource management and system management, and the functions of the management end 102 can be implemented through each sub-module in the management module as follows:
The user management sub-module is used for managing all users logged in the system, wherein the users can be students, teachers and monitoring personnel, and the users can be managed by the management end 102 if the users logged in the system are all managed by the management end;
The device management sub-module is configured to manage all hardware devices that access the system, where the hardware devices may be, for example, devices corresponding to the learner terminal 103, devices corresponding to the teacher terminal 101, and so on;
the scene management sub-module is used for managing all training scenes configured in the system;
the point management sub-module is used for managing the points of the trainees logged in by the trainee terminal 103 in the training scene, and when the roles corresponding to the trainees logged in the trainee terminal 103 are trained in the training scene, the corresponding points can be obtained according to the training performance, and the point management sub-module can manage the points;
the treatment management sub-module is used for carrying out full-flow management on the training scheme corresponding to the training scene;
the scheme management sub-module is used for carrying out configuration management on the training scheme corresponding to the training scene;
The resource management sub-module is used for carrying out configuration management on training resources in a training scheme corresponding to the training scene;
and the system management sub-module is used for carrying out configuration management on the log and the authority of the system.
It should be noted that, the management end 102 also has the function of the server, and may convert the training data of the teacher end 101, the learner end 103 and the monitor end 104, for example, may receive an instruction of the teacher end, obtain corresponding training data from the learner end 103 according to the instruction of the teacher end 101, and convert the obtained training data into a data format that can be identified by the teacher end, or a unified data format defined in advance, and send the converted data to the teacher end. In addition, the management end 102 may be deployed inside a local area network, or may be deployed in a cloud, which is not limited in particular. For example, the user management sub-module may employ a database to store login information for all users, such as user names, passwords, and the like. The device management submodule can be connected with hardware devices through a network interface, such as a VR head display, a handle and the like, so that normal operation and data transmission of the devices are ensured. The scene management submodule is convenient and quick to configure training scenes through a preset scene template, such as field rescue, operating rooms and the like. The point management sub-module may set point rules such as completing a task to obtain points, misoperating to subtract points, etc. The sub-modules are matched with each other, for example, the integral management sub-module can be linked with the user management sub-module to update integral changes to user information in real time.
It should be noted that, the management end 102 may divide the training scenario into different phases, and store the currently completed phase, and then execute the subsequent phase when continuing to train next time, or perform the reinforcement training only for one of the phases.
Referring to fig. 2, fig. 2 is a schematic diagram of an interface of a management end 102 according to an embodiment of the present invention, including user management, device management, scene management, scheme management, resource management, assessment management, and system management.
In one embodiment, the disposition management submodule is specifically configured to perform the following operations:
Configuring a corresponding AI intelligent agent for each of a plurality of treatment procedures corresponding to the training scheme, wherein the plurality of procedures comprise using instruments, using consumables, using medicines, checking parts, executing actions, playing voices, playing sound effects, playing animations, voice recognition, starting procedures, evaluating operations, equipment values, chest compressions, consciousness judgment and opening airways;
Controlling a target AI (advanced technology attachment) agent to execute a target treatment process and acquiring treatment data corresponding to the target treatment process, wherein the target AI agent is an agent corresponding to the target treatment process in the AI agents, and the target treatment process is any one treatment process in the AI agents;
and executing corresponding operation according to the treatment data.
In this embodiment, the treatment management sub-module may classify and separate the instruments, actions, consumables and the like that may be used in all training schemes to form individual prop units (refer to fig. 3, fig. 3 is a schematic diagram of an equipment management interface provided in an embodiment of the present invention, where the configuration of prop units is included), so that each prop unit may be arbitrarily selected when editing a case by an editing end, to form a combined effect, and the case may be automatically loaded with prop units as required and displayed in the case, where the prop may be, for example, a neck brace, a right leg brace, a tripod, a red card, and the like.
In order to realize dynamic configuration, the treatment management submodule is respectively configured with corresponding AI agents for each treatment flow in a plurality of treatment flows corresponding to a training scheme, namely, the treatment management submodule is designed to use 15 AI agents such as instruments, consumables, medicines, part examination, execution actions, voice playing, sound effect playing, animation playing, voice recognition, starting flow, evaluation operation, equipment value, chest compression, consciousness judgment, air passage opening and the like, the whole scheme is automatically divided into a plurality of AI agents according to flow setting, and each agent automatically executes and judges whether the treatment flow corresponding to the agent is correct, how much of the treatment flow is used, which role is executed, whether the execution sequence is correct, what the product results are, and the like.
The prop unit also has AI intelligence, and each device can judge whether the operation flow is correct, automatically display data, operate buttons and the like, and the device can judge and react to the inside of the whole system. Taking the prop as a monitor for example, whether the monitor is started up, whether the lead, the cuff, the oximeter and the like are connected with the monitor correctly or not will be described.
It should be noted that, the configuration of the training solution includes basic information, medical care configuration, wounded configuration, preparation stage task configuration, treatment stage task configuration, transfer stage task configuration, handover stage task configuration, three-dimensional configuration, scene interaction device configuration, and evaluation configuration, and for convenience of understanding, the configuration of the training solution is described in detail below with reference to fig. 4 to 10:
Referring to fig. 4, fig. 4 is a schematic configuration diagram of basic information in training scheme configuration provided in an embodiment of the present invention, where the configuration of basic information includes configuration of a scheme name, a scheme duration, a scheme scene, a scheme description, a voice description, and a scene picture, and further includes configuration of keywords (such as fire) of an newly added emergency;
Referring to fig. 5, fig. 5 is a schematic configuration diagram of a medical configuration in a training scheme configuration according to an embodiment of the present invention, where the medical configuration includes a configuration of adding character types, deleting character types, and diagonal related information.
Referring to fig. 6, fig. 6 is a schematic diagram of a patient configuration in the training scenario configuration provided in the present invention, where the patient configuration includes a configuration of a patient's name, gender, age, patient description and past medical history, such as "patient 1, gender, male, age 22, patient description is cardiac arrest". In addition, more injury cases can be randomly combined based on injuries of various parts.
Referring to fig. 7, fig. 7 is a schematic diagram of a configuration of a preparation task stage in a training scheme configuration provided in an embodiment of the present invention, where the configuration of the preparation task stage includes configuration of time of the preparation stage, task of the preparation stage, treatment type, executor, presence or absence of a pre-task, serial number of the pre-task, presence or absence of a post-task, learning number of the post-task, and the like, and of course, operations of editing, adjusting the serial number, deleting, and the like may also be performed on the task of the preparation stage according to actual situations or user requirements. It will be appreciated that the rescue task phase configuration, the transfer phase task configuration, and the handover phase task configuration are similar to the preparation task phase configuration, and the preparation task phase configuration is described herein as an example, and the task configuration of the other phases may be referred to as the preparation phase task configuration.
Referring to fig. 8, fig. 8 is a schematic diagram of three-dimensional configuration in a training scheme configuration provided by an embodiment of the present invention, where the three-dimensional configuration includes a configuration for a prop type, a prop name, a thumbnail, position information, rotation information, and scaling information, and each prop may be configured according to an actual situation.
Referring to fig. 9, fig. 9 is a schematic diagram of a scene interaction device configuration in a training scheme configuration provided by an embodiment of the present invention, where the scene interaction device configuration includes configuration performed on a task number, a task name, a task description, a response property type, a property thumbnail, a device name, a device number, a device type, and the like.
Referring to fig. 10, fig. 10 is a schematic diagram of an evaluation configuration in a training scheme configuration according to an embodiment of the present invention, where the evaluation configuration includes a configuration of the evaluation content and the score.
In one embodiment, the conversion module is specifically configured to perform the following operations:
receiving target instructions and/or transmission data of the teacher end 101, the student end 103 or the monitoring end 105;
Determining a target end corresponding to the target instruction and/or the transmission data, wherein the target end is the teacher end 101, the student end 103 or the monitoring end 105;
And sending the target instruction to the target end, and/or sending the transmission data to the target end after conversion.
In this embodiment, the conversion module may receive the target instruction and/or transmission data of the teacher end 101, the learner end 103 or the monitor end 105, that is, when the teacher end needs to view the data of the learner end, the management end may send an instruction to the management end, after receiving the instruction through the conversion module, the management end may determine the learner end corresponding to the instruction and send the instruction to the learner end, after receiving the instruction, the learner end returns the data corresponding to the instruction to the conversion module, where the conversion module converts the data to convert the data into data that can be viewed or identified by the teacher end and sends the converted data to the teacher end, and therefore, when the teacher end needs to view the data of the learner end or wants to let a certain learner in the learner end perform the designated operation, only one instruction needs to be sent.
In one embodiment, the learner terminal 103 is specifically configured to perform the following operations;
Downloading training resources from the management end 102 according to the training scheme, wherein the training resources comprise scene resources, role resources and task resources;
Initializing a training scene corresponding to the training scheme;
Assigning simulated task roles to the learner terminal 103, wherein the task roles include a primary doctor, an anesthesiologist, a surgical aid, an instrument nurse, and a tour nurse;
And monitoring vital sign data of the task roles in real time, and sending the vital sign data to the management end 102.
Referring to fig. 11, fig. 11 is a schematic diagram of an interface of a student end provided by an embodiment of the present invention, fig. 12 is another schematic diagram of an interface of a student end provided by an embodiment of the present invention, fig. 11 includes wounded information, equipment information, consumable information and drug information corresponding to a training scene, a specific situation and a text description of the wounded are shown in a virtual picture form, fig. 12 illustrates that the student end includes 4 roles, and from 4 views, a view, B view, C view and L view are illustrated as examples, each view corresponds to an area, and the area includes task countdown and equipment corresponding to the roles.
Specifically, the learner terminal 103 is VR software developed by using a virtual reality technology, and the learner terminal 103 is mainly installed in VR equipment, and when each training is started, training resources including but not limited to scene resources, role resources and task resources are automatically downloaded from the management terminal 102 according to configuration of a training scheme, and enter a training scene after scene initialization.
Each learner side simulates the role assigned by a training regimen, and the default optional system roles are five, namely, a doctor of a primary knife, an anesthesiologist, a surgical aid, an instrument nurse and a nurse for tour, and the roles can be increased or decreased through the management side 102.
The operation scene environment in the training scene of the learner terminal 103 may include, for example, a battlefield environment (including various special effects such as explosion and blood light), a battlefield operating room, a patient, an instrument, a consumable, a medicine, a wounded person, and a dialogue voice, which may be, for example, a simulation dialogue between a medical staff and a patient set in advance, that is, a student playing a role as a doctor needs to perform corresponding output according to the dialogue voice.
The learner terminal 103 is a multi-person networking system, and multiple persons synchronize all operations in real time in the same training scene, where the operations include a device position, a device state, a patient state, a treatment result, an action of each role, a position of each role, and the like of a device corresponding to the learner terminal.
The student end 103 is further configured with an intelligent sensing device (for collecting heart rate, blood oxygen, myoelectric equipment and the like), vital sign data of each role can be collected and monitored in real time through the intelligent sensing device, after vital sign data of a task role corresponding to the student end 103 is monitored, the management end 102 can also determine a target training scheme corresponding to the student end 103, and training and judging psychological adaptability and compressive capacity of the task role according to the vital sign data and the target training scheme, namely, the management end can conduct targeted training according to the performance of a student logged in the student end 103 and the vital sign data of the student in the training process, errors in actual operation are avoided, and psychological adaptability and compressive capacity of the student are enhanced.
It should be noted that, the student ends can form a local area network environment by self-networking, so that synchronization and transmission of large data volume and high real-time data among the student ends can be realized, and the number of monitors, the data of the monitors and the like can be synchronized. In addition, when the number of characters required for training the scene exceeds the login number of the trainee, that is, the number of personnel at the trainee is too small, the characters required for training the scene can be replaced by the AI when no personnel are logged in.
It should be noted that a dummy may be further provided, where the dummy is a dummy that simulates a real person in reality, and the dummy is associated with a patient character, and the dummy is communicated with the patient data, and the vital sign data of the patient character is adjusted by adjusting the vital sign data corresponding to the dummy, that is, when the vital sign data of the dummy changes, the vital sign data of the patient character in the training scene also changes accordingly.
In one embodiment, the teacher end 101 is specifically configured to perform the following operations:
newly building a training scene;
configuring training personnel and training schemes corresponding to the training scene;
Pairing the equipment of the training personnel corresponding to the training scene;
Checking the training progress corresponding to the training scene;
and watching the training visual angle of the training personnel corresponding to the training scene, and controlling whether the training scene is retrained.
Referring to fig. 13, fig. 13 is a schematic diagram of an interface of a teacher end provided by an embodiment of the present invention, where the teacher end includes information of a current training scheme, such as an assessment time, a participant, an assessment progress, and an assessment duration, and also includes information of a device state, a simulator state, and the like, specifically:
The teacher end 101 mainly has teacher operations corresponding to training scenes, and is mainly used for controlling the whole training process, and the teacher end can be installed at the PAD end or the computer end, and has the main functions of newly building training, configuring training personnel and training schemes, pairing equipment (equipment belonging to the same training task is paired), checking training progress (treatment results, vital signs, patient states, total progress and the like of each character), watching visual angles of training personnel, and controlling training (for example, notifying the character to retrain for a certain training scheme, ending training or retraining and the like).
In one embodiment, the editing end 104 is specifically configured to perform the following operations:
Editing the training scene;
editing the number of wounded persons in the training scene, the injury, posture, dressing and consumable material corresponding to wound treatment of each wounded person and the state corresponding to the wound treatment.
In this embodiment, the editing end 104 mainly includes two modules of editing a training scenario, i.e., dynamic editing of a training scenario and dynamic editing of a wounded person corresponding to the training scenario, where the editing of the training scenario includes basic information, medical care configuration, wounded person configuration, preparation stage task configuration, treatment stage task configuration, transfer stage task configuration, handover stage task configuration, three-dimensional configuration, scene interaction device configuration, and evaluation configuration, the dynamic editing of the wounded person includes the number of wounded persons, the wounded condition, physical state, dressing, and which consumables and wound states of the wounded person are used for wound treatment, and, specifically, as shown in fig. 14, fig. 14 is an interface schematic diagram of the editing end provided in the embodiment of the present invention, for example, a category 1301 in fig. 14 may select an area where the wounded person needs to be adjusted, such as a "face, mouth, nose, and face modeling", for example, a wound 1302 may select a wound shape corresponding to the wounded person, and use consumables including but not limited to medicine, gauze, and the wound may select a wound, and the wound related to the wound, and the wound may be adjusted according to the requirements of the user, and the wound location may be adjusted, and the position may be adjusted.
In one embodiment, the monitoring terminal 105 is specifically configured to perform the following operations:
Monitoring a picture corresponding to the student end 103 and vital sign data of a student corresponding to the student end 103;
And storing the monitored video.
Referring to fig. 15, fig. 15 is an interface schematic diagram of a monitoring end provided by an embodiment of the present invention, where the monitoring end 105 is divided into a plurality of viewing angles, each viewing angle corresponds to a picture of a learner at the learner end, the monitoring end 105 can monitor the picture of each learner and vital sign data of each learner when playing its corresponding role, and the monitoring end 105 can also save video in real time when monitoring the learner end, so that a teacher can better decode the training process and multiplex the training process through video playback.
In summary, it can be seen that in the embodiment of the present invention, by the cooperative work of the teacher side, the management side, the learner side, the editing side, and the monitoring side, efficient management and real-time monitoring of the virtual training system are realized. The teacher end is responsible for creating and managing training scenes, the management end is responsible for managing and converting data, the learner end is responsible for executing training processes and collecting data, the editing end is responsible for editing training scenes and wounded information, and the monitoring end is responsible for monitoring states of the learner and recording and storing the information. The functional modules are tightly matched, so that the whole virtual training system can run efficiently and stably, and training effect and student experience are greatly improved.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that modifications may be made to the technical solutions described in the foregoing embodiments or equivalents may be substituted for some of the technical features thereof, and that such modifications or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention in essence.