技术领域technical field
本发明涉及一种航空航天领域的仿真平台开发问题，是验证有人/无人机集群编队控制方法的仿真平台开发问题。具体讲,涉及基于脑电设备的有人/无人机集群编队VR仿真系统。The invention relates to a simulation platform development problem in the field of aerospace, and is a simulation platform development problem for verifying a manned/unmanned aerial vehicle cluster formation control method. Specifically, it involves a human/unmanned aerial vehicle group formation VR simulation system based on EEG equipment.
背景技术Background technique
现代空战中，随着战场环境和作战任务的日趋复杂，仅凭单架无人机或者有人机将无法满足作战需求，有人机与无人机的协同作战将是未来空战的一种重要形式。有人/无人机协同的决策分配就是研究有人机和无人机在不同情况下的决策控制等级问题。将人的经验、直觉、灵活等优势与计算机数据处理能力强、速度快的优势相结合，使两者优势互补，达到更优的决策效果。有人机作为长机带领无人机进行混合编队飞行，这样不仅将使系统的综合效能和任务执行时的冗余性能得到大幅提升，而且弥补了单纯无人机编队对复杂环境识别精度不高和判断困难等方面的不足，此问题成为有人/无人机混合编队及其应用技术发展的研究新热点，目前在编队队形控制、协同作战、信息交互处理以及具体实现验证等多方面已经开展了广泛的研究。随着研究的不断深入，设定的系统运行环境和编队模型的相异性，对验证方法和验证手段带来了更大的挑战，由于实体验证需要完备的工程设计，繁杂的验证工作，大量的人力资源，冗长的工作周期，以及昂贵的实验仪器和物质消耗。因此，虚拟仿真验证作为一种灵活、低耗的验证方法适用于大多数的理论性预研。In modern air combat, as the battlefield environment and combat tasks become more and more complex, a single UAV or manned aircraft will not be able to meet the operational needs. Coordinated operations between manned and UAVs will be an important form of air combat in the future. The decision-making assignment of manned/unmanned aerial vehicles is to study the decision-making control level of manned and unmanned aerial vehicles in different situations. Combining the advantages of human experience, intuition, and flexibility with the advantages of strong data processing ability and fast speed of the computer, the advantages of the two complement each other to achieve better decision-making results. The manned aircraft leads the UAVs to fly in mixed formations as the lead aircraft, which will not only greatly improve the overall performance of the system and the redundancy performance during task execution, but also make up for the low recognition accuracy of the simple UAV formation in complex environments. Judgment difficulties and other deficiencies, this issue has become a new research hotspot in the development of manned/unmanned aerial vehicle mixed formation and its application technology. At present, it has been carried out in many aspects such as formation control, coordinated operations, information interaction processing, and specific implementation verification. Extensive research. With the deepening of the research, the set system operating environment and the dissimilarity of the formation model have brought greater challenges to the verification methods and methods. Because the physical verification requires complete engineering design, complicated verification work, and a large number of Human resources, lengthy work cycles, and expensive experimental equipment and material consumption. Therefore, as a flexible and low-cost verification method, virtual simulation verification is suitable for most theoretical pre-research.
有人/无人机编队控制系统不同与普通通航天飞行器，其系统的功能是由多个飞行器互相协同完成，因此存在着复杂的信息交互。这对飞行器位置姿态控制及飞行器任务管理、调度控制提出了新的要求。基于单一领域仿真模型集成的方法不利于系统的扩展，集成后系统仿真结果置信度低；基于统一建模语言方法需要对已有模型重新用数学方程描述，无法充分利用已有研究成果，并且有些模型无法用方程描述；基于异构仿真环境可以实现仿真系统灵活构建,有效提高仿真效率，集成现有研究成果；基于分布式建模与仿真方案具有一定的灵活性、可扩展性，减少网络数据冗余，最大限度保证数据一致性，并可将真实仿真、虚拟仿真等集成到统一的仿真环境中。The manned/unmanned aerial vehicle formation control system is different from ordinary space vehicles. Its system functions are completed by multiple aircraft in cooperation with each other, so there is complex information interaction. This puts forward new requirements for aircraft position and attitude control, aircraft task management, and dispatch control. The method based on the integration of simulation models in a single field is not conducive to the expansion of the system, and the confidence of the system simulation results after integration is low; the method based on the unified modeling language needs to describe the existing models with mathematical equations again, which cannot make full use of the existing research results, and some The model cannot be described by equations; based on the heterogeneous simulation environment, the flexible construction of the simulation system can be realized, which can effectively improve the simulation efficiency and integrate existing research results; based on the distributed modeling and simulation scheme, it has certain flexibility and scalability, and reduces network data Redundancy ensures maximum data consistency, and can integrate real simulation, virtual simulation, etc. into a unified simulation environment.
有人/无人编队仿真平台采用计算机实时仿真、嵌入式软硬件技术和虚拟现实技术实现有人/无人编队的控制方法仿真与验证，其最大的特色在于仿真模型可以快速高效的移植到平台上，并在虚拟现实环境中实时观看仿真效果。符合有人/无人编队仿真的应用需求和发展趋势。The manned/unmanned formation simulation platform uses computer real-time simulation, embedded software and hardware technology, and virtual reality technology to realize the simulation and verification of manned/unmanned formation control methods. Its biggest feature is that the simulation model can be quickly and efficiently transplanted to the platform. And watch the simulation effect in real time in the virtual reality environment. It meets the application requirements and development trends of manned/unmanned formation simulation.
通过对现有技术的检索，并未发现类似专利。特别是针对有人/无人编队控制算法实时验证方面，没有一种有效的仿真验证平台。Through searching the prior art, no similar patents were found. Especially for the real-time verification of manned/unmanned formation control algorithms, there is no effective simulation verification platform.
发明内容Contents of the invention
为克服现有技术的不足，本发明旨在提出一种能够支持有人/无人机集群编队控制算法验证的仿真平台。本发明采用的技术方案是，基于脑电的有人/无人机集群编队VR仿真系统，实时仿真Simulink Real Time目标机负责Matlab仿真程序的实时仿真功能，通过以太网与仿真平台管理计算机连接，仿真平台管理计算机将MATLAB程序下载到目标机中，目标机运行MATLAB程序，并将仿真结果实时发送给仿真平台管理计算机，并在该计算机上进行显示和数据保存；每台Simulink Real Time目标机+嵌入式控制器代表一台无人机，目标机运行无人机数字仿真simulink模型；嵌入式控制器运行无人机控制算法，计算出控制信号，并通过串口传递给目标机，实现无人机控制回路；脑电设备+人机交互装置模拟有人机，脑电装置采集实验员脑信号，转化为相应的编队控制指令，模拟有人机飞行员操作，人机交互手柄接收人的手势动作，模拟飞行员的对有人机的操作；仿真平台管理计算机接收这两个控制信号并将有人机指令发送给操控无人机；视景计算机接收仿真产生的位置、姿态角信息，在大屏幕显示系统上进行实时视景演示。In order to overcome the deficiencies of the prior art, the present invention aims to propose a simulation platform that can support the verification of manned/unmanned aerial vehicle swarm formation control algorithms. The technical scheme that the present invention adopts is, based on EEG manned/unmanned aerial vehicle cluster formation VR simulation system, real-time simulation Simulink Real Time target machine is responsible for the real-time simulation function of Matlab simulation program, is connected with simulation platform management computer by Ethernet, simulation The platform management computer downloads the MATLAB program to the target machine, and the target machine runs the MATLAB program, and sends the simulation results to the simulation platform management computer in real time, and displays and saves data on the computer; each Simulink Real Time target machine + embedded The embedded controller represents a UAV, and the target machine runs the UAV digital simulation simulink model; the embedded controller runs the UAV control algorithm, calculates the control signal, and transmits it to the target machine through the serial port to realize the control of the UAV Loop; EEG equipment + human-computer interaction device to simulate a manned machine. The EEG device collects the brain signal of the experimenter and converts it into a corresponding formation control command to simulate the operation of the manned machine pilot. The operation of the manned machine; the simulation platform management computer receives the two control signals and sends the manned machine command to the unmanned aerial vehicle; the vision computer receives the position and attitude angle information generated by the simulation, and performs real-time viewing on the large-screen display system. scene demo.
平台管理软件运行在平台管理计算机上，包含数字仿真模块、实时仿真模块、数据管理模块、网络通信模块、脑机接口模块和人机交互模块，具体如下：The platform management software runs on the platform management computer, including digital simulation module, real-time simulation module, data management module, network communication module, brain-computer interface module and human-computer interaction module, as follows:
(1)数字仿真模块：建立仿真软件Matlab/Simulink环境下可复用的数字仿真回路，并基于模块化的思想将数字仿真回路分为有人/无人机模型库、编队队形库、任务分配方法库、轨迹优化方法库和编队控制方法库五个模块，其中，有人/无人机模型库提供了包括四旋翼、六旋翼飞机模型，编队队形库提供了包括攻击队形数据、侦查队形数据、巡航队形数据、任务分配方法库提供了包括群算法、市场类算法、聚类算法的动态和静态任务分配算法；轨迹优化方法库提供包括自适应遗传算法、改进粒子群算法的轨迹优化方法；编队控制方法库提供包括PID，滑模、反步的编队控制方法，在一个可复用的数字回路中，在不同的库中选择不同的模型、编队队形、任务分配方法、轨迹优化方法以及控制算法，生成数字仿真dlm文件；在进行回路模块化设计的过程中，统一模块之间进行交互的I/O接口，使其具有通用性，为后续开发和扩展提供方便；(1) Digital simulation module: establish a reusable digital simulation loop under the simulation software Matlab/Simulink environment, and divide the digital simulation loop into manned/unmanned aerial vehicle model library, formation library, and task assignment based on the idea of modularization There are five modules: method library, trajectory optimization method library and formation control method library. Among them, the manned/unmanned aerial vehicle model library provides quadrotor and hexacopter aircraft models, and the formation formation library provides attack formation data, reconnaissance team Shape data, cruise formation data, and task allocation method library provide dynamic and static task allocation algorithms including swarm algorithm, market algorithm, and clustering algorithm; trajectory optimization method library provides trajectory including adaptive genetic algorithm and improved particle swarm algorithm Optimization method; the formation control method library provides formation control methods including PID, sliding mode, and backstepping. In a reusable digital loop, different models, formation formations, task assignment methods, and trajectories can be selected in different libraries Optimize the method and control algorithm to generate digital simulation dlm files; in the process of loop modular design, unify the interactive I/O interface between modules to make it universal and provide convenience for subsequent development and expansion;
(2)实时仿真模块：基于Simulink Real Time函数库解析上述五个模块构成的数字仿真模型，实现仿真数据监控，并开发模型连接、程序下载、开始仿真、结束仿真功能模块，管理集群编队实时仿真全过程；实时仿真数据显示则基于绘图控件Teechart开发曲线绘制子模块，实现对于有人/无人机编队实时位置信息、姿态角、速度的关键数据的显示；(2) Real-time simulation module: Analyze the digital simulation model composed of the above five modules based on the Simulink Real Time function library, realize simulation data monitoring, develop model connection, program download, start simulation, end simulation function modules, and manage cluster formation real-time simulation The whole process; the real-time simulation data display is based on the drawing control Teechart to develop the curve drawing sub-module to realize the display of the key data of the real-time position information, attitude angle and speed of the manned/unmanned aerial vehicle formation;
(3)数据管理模块：基于数据库Mysql提供的函数开发包API，开发数据管理子模块，实现对1组有人/无人机编队实时仿真数据的管理功能，包含数据存储、数据修改、数据删除、数据对比、数据导出功能模块，充分利用仿真过程产生的数据；(3) Data management module: Based on the function development kit API provided by the database Mysql, develop a data management sub-module to realize the management function of real-time simulation data of a group of manned/drone formations, including data storage, data modification, data deletion, Data comparison and data export function modules make full use of the data generated during the simulation process;
(4)网络通信模块：基于用户数据报协议UDP开发网络通信模块，将有人/无人机编队的实时仿真数据传输给视景演示软件，驱动视景演示软件进行动态显示；(4) Network communication module: develop a network communication module based on the User Datagram Protocol UDP, transmit the real-time simulation data of manned/unmanned aerial vehicle formation to the visual demonstration software, and drive the visual demonstration software for dynamic display;
(5)脑机接口模块：包含信号采集子模块和信号转换子模块，基于脑电采集系统Neuracle采集用户脑电信号。基于脑电设备配套软件开发单元NeuroCube，采用Neuracle配套软件开发，转换为相应有人机控制指令，实时发送给无人机控制器，完成基于脑控的有人/无人机集群虚拟飞行验证；(5) Brain-computer interface module: includes a signal acquisition sub-module and a signal conversion sub-module, and collects user's EEG signals based on the EEG acquisition system Neuracle. NeuroCube, a supporting software development unit based on EEG equipment, is developed with Neuracle supporting software, converted into corresponding manned-machine control commands, and sent to the UAV controller in real time to complete the virtual flight verification of manned/drone clusters based on brain control;
(6)人机交互模块：包含手势识别子模块。基于手势识别技术Orion，识别操作者动作，并将其转化为相应的有人机控制指令，控制无人机编队飞行，进行有人/无人协同飞行仿真验证，实现“人在回路”仿真。(6) Human-computer interaction module: including gesture recognition sub-module. Based on the gesture recognition technology Orion, it recognizes the operator's actions and converts them into corresponding manned-machine control commands, controls the formation flight of the drones, conducts manned/unmanned collaborative flight simulation verification, and realizes "human-in-the-loop" simulation.
视景演示软件运行在视景计算机中：设计三维虚拟场景模块、人机交互模块和网络通信模块，具体如下：The visual demonstration software runs in the visual computer: design the 3D virtual scene module, the human-computer interaction module and the network communication module, as follows:
(1)三维虚拟场景模块：包含VR模型、场景、特效的设计与实现，通过三维建模软件SketchUp、3D max建模，建立包含多种固定翼、旋翼的有人/无人机模型库，及山地、平原、森林环境下的陆地场景库，并将其导入图形开发引擎Unity建立应用程序资源组，基于物理引擎Physics实现集群编队对军事目标打击时产生的爆炸、碎片特效，基于Unity跨平台发布的功能，将虚拟场景输出至VR头显设备中显示；(1) 3D virtual scene module: including the design and realization of VR models, scenes, and special effects. Through 3D modeling software SketchUp and 3D max modeling, a manned/unmanned aerial vehicle model library including various fixed wings and rotors is established, and The land scene library in the mountains, plains, and forest environments, and import it into the graphics development engine Unity to establish an application resource group, based on the physics engine Physics to realize the special effects of explosions and fragments generated when the cluster formation strikes military targets, and release it across platforms based on Unity The function of outputting the virtual scene to the VR head display device for display;
(2)人机交互模块：包含手势识别子模块、用户图形界面GUI子模块、头部追踪子模块，基于Orion手势识别技术，实现用户通过手势在VR视景演示中，与程序进行交互的功能，基于三维图形界面库Hovercast设计VR场景中的GUI界面，实现用户对仿真信息的查看、沙盘演示、场景配置；基于头部追踪组件OpenVR实现VR视角切换功能；(2) Human-computer interaction module: including gesture recognition sub-module, user graphical interface GUI sub-module, head tracking sub-module, based on Orion gesture recognition technology, realizes the user's function of interacting with the program in the VR visual demonstration through gestures , based on the three-dimensional graphics interface library Hovercast to design the GUI interface in the VR scene, to realize the user's viewing of simulation information, sand table demonstration, and scene configuration; based on the head tracking component OpenVR to realize the VR viewing angle switching function;
(3)网络通信模块：基于UDP协议开发网络通信模块，接收编队的实时仿真数据，驱动虚拟场景中有人/无人机的运动，演示有人/无人编队队形生成和保持的过程。(3) Network communication module: develop a network communication module based on the UDP protocol, receive real-time simulation data of the formation, drive the movement of manned/unmanned aerial vehicles in the virtual scene, and demonstrate the formation and maintenance process of manned/unmanned formations.
基于脑电的有人/无人机集群编队VR仿真方法，利用Simulink Real Time目标机负责Matlab仿真程序的实时仿真功能，通过以太网与仿真平台管理计算机连接，仿真平台管理计算机将MATLAB程序下载到目标机中，目标机运行MATLAB程序，并将仿真结果实时发送给仿真平台管理计算机，并在该计算机上进行显示和数据保存；每台Simulink RealTime目标机+嵌入式控制器代表一台无人机，目标机运行无人机simulink模型；嵌入式控制器运行无人机控制算法，计算出控制信号，并通过串口传递给目标机，实现无人机控制回路；脑电设备+人机交互装置模拟有人机，脑电装置采集实验员脑信号，转化为相应的编队控制指令，模拟有人机飞行员操作，人机交互手柄接收人的手势动作，模拟飞行员的对有人机的操作；仿真平台管理计算机接收这两个控制信号并将有人机指令发送给操控无人机；视景计算机接收仿真产生的位置、姿态角信息，在大屏幕显示系统上进行实时视景演示。EEG-based manned/unmanned aerial vehicle cluster formation VR simulation method, using Simulink Real Time target machine to be responsible for the real-time simulation function of the Matlab simulation program, connected to the simulation platform management computer through Ethernet, and the simulation platform management computer downloads the MATLAB program to the target In the machine, the target machine runs the MATLAB program, and sends the simulation results to the simulation platform management computer in real time, and displays and saves data on the computer; each Simulink RealTime target machine + embedded controller represents a UAV, The target machine runs the UAV simulink model; the embedded controller runs the UAV control algorithm, calculates the control signal, and transmits it to the target machine through the serial port to realize the UAV control loop; EEG equipment + human-computer interaction device simulates a human The EEG device collects the brain signals of the experimenters, converts them into corresponding formation control commands, and simulates the operation of manned-machine pilots. The human-computer interaction handle receives human gestures and simulates the pilot's operation of manned machines; the simulation platform management computer receives these Two control signals and man-machine commands are sent to the UAV; the vision computer receives the position and attitude angle information generated by the simulation, and performs real-time vision demonstration on the large-screen display system.
本发明的特点及有益效果是：Features and beneficial effects of the present invention are:
社会效益：此项发明对有人/无人机编队控制系统的研究具有十分重要的意义。该项发明具有国际先进水平，它不仅可以提高有人/无人机编队控制系统开发初期的仿真可信度，大大缩短研究周期；同时又降低了平台开发成本，并支持对不同编队控制方法的仿真验证，有效提高了有人/无人机编队控制的理论研究水平和仿真试验水平，为有人/无人机控制系统的研究与发展打下良好的试验平台基础。Social benefits: This invention is of great significance to the research of manned/unmanned aerial vehicle formation control system. This invention is at the international advanced level. It can not only improve the simulation reliability in the early stage of manned/unmanned aerial vehicle formation control system development, and greatly shorten the research cycle; at the same time, it reduces the platform development cost and supports the simulation of different formation control methods. The verification has effectively improved the theoretical research level and simulation test level of manned/unmanned aerial vehicle formation control, and laid a good test platform foundation for the research and development of manned/unmanned aerial vehicle control system.
经济效益：有人/无人机编队结合了有人机和无人机作战的优势，能够简化作战任务，扩大观测领域，提高系统健壮性，还可以根据任务的变更替换系统载荷，提高有人/无人机编队对任务的适应性，具有较高的经济价值，在军事方面都具有很大的潜在应用。该仿真与验证平台针对有人/无人机编队控制系统进行设计，不仅可以为未来有人/无人机编队飞行控制系统开发提供更加全面、复杂的仿真验证功能；同时可以作为研究有人/无人机编队飞行控制系统的快速原型化平台，解决仿真优化设计及控制策略验证问题，既快速高效又节省开支。Economic benefits: The manned/unmanned aerial vehicle formation combines the advantages of manned and unmanned aerial vehicles, which can simplify combat tasks, expand the observation field, and improve system robustness. It can also replace system loads according to task changes, improving manned/unmanned The adaptability of aircraft formations to tasks has high economic value and has great potential applications in military. The simulation and verification platform is designed for the manned/unmanned aerial vehicle formation control system, which can not only provide more comprehensive and complex simulation verification functions for the development of the future manned/unmanned aerial vehicle formation flight control system; The rapid prototyping platform of the formation flight control system solves the problems of simulation optimization design and control strategy verification, which is fast, efficient and cost-effective.
附图说明：Description of drawings:
图1有人/无人机编队仿真平台硬件结构示意图。Fig. 1 Schematic diagram of the hardware structure of the manned/unmanned aerial vehicle formation simulation platform.
图2基于脑电设备的有人机与无人机的协同仿真方法流程图。Fig. 2 is a flow chart of the co-simulation method of manned machine and unmanned aerial vehicle based on EEG equipment.
图3平台管理软件软件结构示意图。Figure 3 is a schematic diagram of the software structure of the platform management software.
图4视景演示软件结构图。Figure 4 Structural diagram of the visual demonstration software.
图5Simulink Real Time编程实现流程图。Figure 5 Simulink Real Time programming implementation flow chart.
图6Simulink Real Time target串口通信流程图。Figure 6 Simulink Real Time target serial communication flow chart.
具体实施方式detailed description
本发明的目的在于提供一种能够支持有人/无人机集群编队控制算法验证的仿真平台。The purpose of the present invention is to provide a simulation platform that can support the verification of manned/unmanned aerial vehicle swarm formation control algorithm.
本发明充分利用Simulink Real Time实时仿真技术构建实时仿真模块，基于Simulink Real Time宿主机——目标机架构，该技术通过PC机构建有人/无人编队动力学仿真环境，同时结合无缝连接Matlab的优势，解决了传统数字仿真不能在线调整参数的问题，实现了Matlab/Simulink仿真程序的快速C代码化和仿真实时性。有效提高了有人/无人机编队控制的理论研究水平和仿真试验水平，为有人/无人机编队协同控制系统的研究与发展打下良好的试验平台基础。The present invention makes full use of the Simulink Real Time real-time simulation technology to build a real-time simulation module, based on the Simulink Real Time host machine-target machine architecture, this technology builds a manned/unmanned formation dynamics simulation environment through the PC machine, and simultaneously combines the seamless connection with Matlab Advantages, it solves the problem that traditional digital simulation cannot adjust parameters online, and realizes the fast C coding and real-time simulation of Matlab/Simulink simulation programs. It effectively improves the theoretical research level and simulation test level of manned/unmanned aerial vehicle formation control, and lays a good test platform foundation for the research and development of manned/unmanned aerial vehicle formation cooperative control system.
本发明基于嵌入式控制器PcDuino设计有人无人编队控制器模块，采用的Linux嵌入式操作系统支持多线程任务和应用程序的开发，通过Wifi无线网络技术模拟有人/无人编队间通讯，可扩展性强，通讯拓扑结构易重构，本发明的硬件平台支持后期开发各种应用程序。The present invention designs a manned and unmanned formation controller module based on the embedded controller PcDuino. The Linux embedded operating system adopted supports the development of multi-thread tasks and application programs, and simulates communication between manned/unmanned formations through Wifi wireless network technology, which is scalable Strong, easy to reconfigure the communication topology, the hardware platform of the present invention supports later development of various application programs.
本发明基于头显Htc Vive设计有人/无人编队虚拟现实体验环境模块，基于Unity构建有人/无人编队虚拟现实软件，实现对于有人/无人编队仿真的虚拟现实体验。The present invention designs a manned/unmanned formation virtual reality experience environment module based on the head-mounted display Htc Vive, builds a manned/unmanned formation virtual reality software based on Unity, and realizes a virtual reality experience for manned/unmanned formation simulation.
本发明基于Neuracle脑控设备设计有人/无人编队脑控仿真实验模块，根据脑控设备采集的脑信号，将其转化为相应的控制指令。模拟有人机控制指令，进行“人在回路”仿真。The present invention designs a human/unmanned formation brain control simulation experiment module based on the Neuracle brain control equipment, and converts the brain signals collected by the brain control equipment into corresponding control instructions. Simulate human-machine control commands to perform "human-in-the-loop" simulation.
本发明功能与特点如下：Function and characteristics of the present invention are as follows:
(1)仿真平台支持基于Simulink Real Time的实时仿真功能，即在给定有人/无人编队的期望轨迹、姿态参数，动力配置的情况下，能够计算出任意仿真时刻的有人/无人机编队和姿态数据。(1) The simulation platform supports the real-time simulation function based on Simulink Real Time, that is, given the expected trajectory, attitude parameters and power configuration of the manned/unmanned formation, it can calculate the manned/unmanned formation at any simulation moment and attitude data.
(2)仿真系统结构支持有人/无人机集群编队控制算法的仿真验证，该发明设计了飞行控制器模块结构，支持将研究的Simulink模型编译为C代码进行实时仿真验证。(2) The simulation system structure supports the simulation verification of manned/unmanned aerial vehicle swarm formation control algorithm. The invention designs the flight controller module structure and supports compiling the studied Simulink model into C code for real-time simulation verification.
(3)仿真平台支持基于虚拟现实VR技术的有人/无人编队可视化仿真，VR头显提供沉浸式体验环境，实验员通过人机交互装置模拟有人机操作员发出操作指令，直观查看仿真过程中有人/无人编队的飞行视景演示效果。(3) The simulation platform supports the visual simulation of manned/unmanned formations based on virtual reality VR technology. The VR head display provides an immersive experience environment. The experimenter simulates the operation instructions issued by the man-machine operator through the human-computer interaction device to visually view the simulation process. Manned/unmanned formation flight visual demonstration effect.
(4)仿真平台支持基于脑电设备的有人机与无人机的协同仿真，利用脑电设备将脑信号转化为计算机可以识别的电信号，从而模拟有人机控制指令，并通过交互设备模拟有人机操作，实现“人在回路”仿真。(4) The simulation platform supports the co-simulation of man-machine and unmanned aerial vehicle based on EEG equipment, and uses EEG equipment to convert brain signals into electrical signals that can be recognized by computers, thereby simulating man-machine control commands, and simulating man-made through interactive equipment Machine operation to realize "human-in-the-loop" simulation.
本发明的仿真平台，可以提高仿真的真实性和可信度，通过实时仿真和控制器的软硬件结合，增强了系统控制策略的实时性和可移植性，为有人/无人机集群编队控制系统开发节约了成本，也为国内有人/无人编队控制系统的产业化奠定了基础。The simulation platform of the present invention can improve the authenticity and credibility of the simulation, and through the combination of real-time simulation and controller software and hardware, the real-time and portability of the system control strategy are enhanced, and it is a manned/unmanned aerial vehicle cluster formation control System development saves costs and also lays the foundation for the industrialization of domestic manned/unmanned formation control systems.
有人/无人机编队仿真与验证平台，硬件包括实时仿真模块、飞控模块、VR模块、平台管理模块、脑控模块、大屏幕显示模块。软件设计主要包括平台管理软件和视景VR仿真软件。Manned/UAV formation simulation and verification platform, the hardware includes real-time simulation module, flight control module, VR module, platform management module, brain control module, and large-screen display module. Software design mainly includes platform management software and visual VR simulation software.
本发明硬件设计方案如下：The hardware design scheme of the present invention is as follows:
本发明基于Simulink Real Time实时仿真技术进行开发，Simulink Real Time实时仿真目标机主要负责Matlab仿真程序的实时仿真功能，通过以太网与仿真平台管理计算机连接，仿真平台管理计算机将MATLAB程序下载到目标机中，目标机运行MATLAB程序，并将仿真结果实时发送给仿真平台管理计算机，并在该计算机上进行显示和数据保存，便于后期分析处理。每台Simulink Real Time目标机+嵌入式控制器代表一台无人机，目标机运行无人机simulink模型。嵌入式控制器无人机控制算法，计算出控制信号，并通过串口传递给目标机，实现无人机控制回路。脑电设备+人机交互装置模拟有人机。脑电装置采集实验员脑信号，转化为相应的编队控制指令，模拟有人机飞行员操作，人机交互手柄接收人的手势动作，模拟飞行员的对有人机的操作。仿真平台管理计算机接收这两个控制信号并将有人机指令发送给无人机。视景计算机接收仿真产生的位置、姿态角等信息，在大屏幕显示系统上进行实时视景演示。The present invention is developed based on the Simulink Real Time real-time simulation technology, and the Simulink Real Time real-time simulation target machine is mainly responsible for the real-time simulation function of the Matlab simulation program, and is connected with the simulation platform management computer through Ethernet, and the simulation platform management computer downloads the MATLAB program to the target machine In the process, the target machine runs the MATLAB program, and sends the simulation results to the simulation platform management computer in real time, and displays and saves the data on the computer, which is convenient for later analysis and processing. Each Simulink Real Time target machine + embedded controller represents a UAV, and the target machine runs the UAV simulink model. The embedded controller UAV control algorithm calculates the control signal and transmits it to the target machine through the serial port to realize the UAV control loop. EEG equipment + human-computer interaction device simulates man-machine. The EEG device collects the brain signals of the experimenters, converts them into corresponding formation control commands, and simulates the operation of manned-machine pilots. The human-machine interaction handle receives human gestures and simulates the pilot's operation of the manned machine. The simulation platform management computer receives these two control signals and sends man-machine instructions to the UAV. The visual computer receives the position, attitude angle and other information generated by the simulation, and performs real-time visual demonstration on the large-screen display system.
平台管理软件运行在平台管理计算机上，包含数字仿真模块、实时仿真模块、数据管理模块和网络通信模块，技术方案如下：The platform management software runs on the platform management computer, including digital simulation module, real-time simulation module, data management module and network communication module. The technical scheme is as follows:
(1)数字仿真模块：建立Matlab/Simulink环境下可复用的数字仿真回路，并基于模块化的思想将数字仿真回路分为有人/无人机模型库、编队队形库、任务分配方法库、轨迹优化方法库和编队控制方法库五个模块，内置丰富的机体模型、编队队形和编队方法。其中，有人无人机模型库提供了诸如四旋翼、六旋翼等飞机模型，编队队形库提供了诸如攻击队形、侦查队形、巡航队形等队形数据、任务分配方法库提供了诸如群算法、市场类算法、聚类算法等动态和静态任务分配算法，轨迹优化方法库提供诸如自适应遗传算法、改进粒子群算法等轨迹优化方法。编队控制方法库提供PID，滑模、反步等编队控制方法。在一个可复用的数字回路中，在不同的库中选择不同的模型、编队队形、任务分配方法、轨迹优化方法以及控制算法，生成数字仿真dlm文件。在进行回路模块化设计的过程中，统一模块之间进行交互的I/O接口，使其具有通用性，为后续开发和扩展提供方便。(1) Digital simulation module: establish a reusable digital simulation circuit in the Matlab/Simulink environment, and divide the digital simulation circuit into manned/unmanned aerial vehicle model library, formation library, and task allocation method library based on the idea of modularization There are five modules including trajectory optimization method library and formation control method library, with rich built-in airframe models, formation formations and formation methods. Among them, the manned UAV model library provides aircraft models such as quadrotors and six rotors, the formation formation library provides formation data such as attack formation, reconnaissance formation, and cruise formation, and the task allocation method library provides such as Dynamic and static task assignment algorithms such as swarm algorithm, market algorithm, clustering algorithm, etc. The trajectory optimization method library provides trajectory optimization methods such as adaptive genetic algorithm and improved particle swarm algorithm. The formation control method library provides formation control methods such as PID, sliding mode, and backstepping. In a reusable digital loop, different models, formation formations, task allocation methods, trajectory optimization methods and control algorithms are selected in different libraries to generate digital simulation dlm files. In the process of loop modular design, the I/O interface for interaction between modules is unified to make it universal and convenient for subsequent development and expansion.
(2)实时仿真模块：基于Simulink Real Time函数库解析上述模块构成的数字仿真模型，实现仿真数据监控，并开发模型连接、程序下载、开始仿真、结束仿真等功能模块，管理集群编队实时仿真全过程；实时仿真数据显示则基于Teechart控件开发曲线绘制子模块，实现对于有人/无人机编队实时位置信息、姿态角、速度等关键数据的显示。(2) Real-time simulation module: Analyze the digital simulation model composed of the above modules based on the Simulink Real Time function library, realize simulation data monitoring, and develop functional modules such as model connection, program download, start simulation, end simulation, etc., and manage the real-time simulation of cluster formations. process; the real-time simulation data display is based on the Teechart control to develop a curve drawing sub-module to realize the display of key data such as real-time position information, attitude angle, and speed of manned/unmanned aerial vehicle formations.
(3)数据管理模块：基于Mysql数据库提供的API，开发数据管理子模块，实现对1组有人/无人机编队实时仿真数据的管理功能，包含数据存储、数据修改、数据删除、数据对比、数据导出等功能模块，充分利用仿真过程产生的数据。(3) Data management module: Based on the API provided by the Mysql database, develop a data management sub-module to realize the management function of real-time simulation data of a group of manned/unmanned aerial vehicle formations, including data storage, data modification, data deletion, data comparison, Data export and other functional modules make full use of the data generated in the simulation process.
(4)网络通信模块：基于UDP协议开发网络通信模块，利用其资源消耗小、处理速度快的优点，将有人/无人机编队的实时仿真数据传输给视景演示软件，驱动视景演示软件进行动态显示。(4) Network communication module: develop network communication module based on UDP protocol, take advantage of its low resource consumption and fast processing speed, transmit the real-time simulation data of manned/unmanned aerial vehicle formation to the visual demonstration software, and drive the visual demonstration software for dynamic display.
(5)脑机接口模块：包含信号采集子模块和信号转换子模块。基于Neuracle脑电采集系统，采集用户脑电信号。基于NeuroCube单元，采用Neuracle配套软件开发，转换为相应有人机控制指令，实时发送给无人机控制器，完成基于脑控的有人/无人机集群虚拟飞行验证。(5) Brain-computer interface module: including signal acquisition sub-module and signal conversion sub-module. Based on the Neuracle EEG acquisition system, the user's EEG signal is collected. Based on the NeuroCube unit, it is developed with Neuracle supporting software, converted into corresponding manned-machine control commands, and sent to the UAV controller in real time to complete the virtual flight verification of the manned/drone cluster based on brain control.
(6)人机交互模块：包含手势识别子模块。于Orion手势识别技术，识别操作者动作，并将其转化为相应的有人机控制指令，控制无人机编队飞行，进行有人/无人协同飞行仿真验证，实现“人在回路”仿真。(6) Human-computer interaction module: including gesture recognition sub-module. Based on Orion gesture recognition technology, it recognizes the operator's actions and converts them into corresponding manned-machine control commands to control the formation flight of the drones, conduct manned/unmanned collaborative flight simulation verification, and realize "human-in-the-loop" simulation.
视景演示软件运行在视景计算机中：设计三维虚拟场景模块、人机交互模块、网络通信模块和脑电设备模块，技术方案如下：The visual demonstration software runs in the visual computer: design the three-dimensional virtual scene module, the human-computer interaction module, the network communication module and the EEG equipment module. The technical scheme is as follows:
(1)三维虚拟场景模块：包含VR模型、场景、特效的设计与实现。通过SketchUp、3Dmax建模，建立包含多种固定翼、旋翼的有人/无人机模型库，及山地、平原、森林等多种环境下的陆地场景库，并将其导入Unity引擎建立应用程序资源组。基于Physics物理引擎实现集群编队对军事目标打击时产生的爆炸、碎片特效。基于Unity跨平台发布的功能，将虚拟场景输出至VR头显设备中显示。(1) Three-dimensional virtual scene module: including the design and realization of VR models, scenes and special effects. Through SketchUp and 3Dmax modeling, establish manned/unmanned aerial vehicle model libraries including various fixed wings and rotors, and land scene libraries in various environments such as mountains, plains, forests, etc., and import them into the Unity engine to create application resources Group. Based on the Physics physics engine, the special effects of explosions and fragments generated when the cluster formation strikes military targets are realized. Based on the functions released by Unity cross-platform, the virtual scene is output to the VR head-mounted display device for display.
(2)人机交互模块：包含手势识别子模块、GUI界面子模块、头部追踪子模块。基于Orion手势识别技术，实现用户通过手势在VR视景演示中，与程序进行交互的功能。基于Hovercast三维图形界面库设计VR场景中的GUI界面，实现用户对仿真信息的查看、沙盘演示、场景配置等功能；基于OpenVR头部追踪组件，实现VR视角切换功能。(2) Human-computer interaction module: including gesture recognition sub-module, GUI interface sub-module, head tracking sub-module. Based on Orion gesture recognition technology, the user can interact with the program through gestures in the VR visual demonstration. Based on the Hovercast 3D graphic interface library, the GUI interface in the VR scene is designed to realize functions such as user viewing of simulation information, sand table demonstration, and scene configuration; based on the OpenVR head tracking component, the VR viewing angle switching function is realized.
(3)网络通信模块：基于UDP协议开发网络通信模块，接收编队的实时仿真数据，驱动虚拟场景中有人/无人机的运动，演示有人/无人编队队形生成和保持的过程。(3) Network communication module: develop a network communication module based on the UDP protocol, receive real-time simulation data of the formation, drive the movement of manned/unmanned aerial vehicles in the virtual scene, and demonstrate the formation and maintenance process of manned/unmanned formations.
结合附图对本发明作进一步详述。The present invention will be described in further detail in conjunction with the accompanying drawings.
参见图1，为有人/无人机控制方法仿真与验证平台的硬件结构图，主要包括：运行有平台管理软件的仿真主控机，负责实时仿真的Simulink Real Time实时仿真计算机，以及实现编队控制的嵌入式控制器、负责视景演示的视景计算机、提供人机交互功能的感知元件、采集脑电信号的脑控设备等。See Figure 1, which is a hardware structure diagram of the simulation and verification platform for the manned/unmanned aerial vehicle control method, which mainly includes: a simulation master computer running platform management software, a Simulink Real Time real-time simulation computer responsible for real-time simulation, and formation control Embedded controllers, visual computers responsible for visual demonstrations, sensory components that provide human-computer interaction functions, brain-controlled devices that collect EEG signals, etc.
仿真主控机：运行仿真平台管理软件，负责将编队构型控制软件、Matlab/Simulink有人/无人机仿真模型，数据通讯模型等下载到Simulink Real Time实时仿真计算机中，形成分布式结构的有人/无人机编队控制实时仿真环境，同时实现编队飞行控制系统的仿真进程监控和数据库管理等功能。Simulation master computer: run the simulation platform management software, responsible for downloading the formation control software, Matlab/Simulink manned/unmanned aerial vehicle simulation model, data communication model, etc. to the Simulink Real Time real-time simulation computer to form a distributed structure of manned / UAV formation control real-time simulation environment, while realizing the simulation process monitoring and database management functions of the formation flight control system.
Simulink Real Time实时仿真计算机：基于DOS环境，构建实时仿真环境，用以模拟有人/无人机编队模型。仿真机上运行控制模型，将模型信息发送给各编队控制器，同时，控制器将计算后的信息发送给实时仿真计算机，两者构成有人/无人机控制闭环。Simulink Real Time real-time simulation computer: Based on the DOS environment, build a real-time simulation environment to simulate manned/unmanned aerial vehicle formation models. The control model is run on the simulator, and the model information is sent to each formation controller. At the same time, the controller sends the calculated information to the real-time simulation computer. The two form a closed loop of manned/unmanned aerial vehicle control.
嵌入式控制器：基于Linux嵌入式操作系统，运行有人/无人机位置与姿态控制模块，实现有人/无人机编队控制，通过无线Wifi通讯模块接收编队控制指令，计算出控制信号，并通过串口传递给对应的仿真从机，实现有人/无人机编队控制回路。Embedded controller: Based on the Linux embedded operating system, it runs the manned/drone position and attitude control module to realize manned/drone formation control, receives formation control instructions through the wireless Wifi communication module, calculates the control signal, and passes The serial port is transmitted to the corresponding simulation slave to realize the manned/unmanned aerial vehicle formation control loop.
视景仿真计算机：基于Windows嵌入式操作系统，运行有人/无人机编队视景软件，并将视频信号投放到大屏幕及VR头显上，实时显示有人/无人编队飞行状态。Visual simulation computer: Based on Windows embedded operating system, it runs manned/unmanned formation visual software, and puts video signals on the large screen and VR head display to display the flight status of manned/unmanned formation in real time.
VR头显：基于Htc Vive，接收有人/无人编队飞行视频信号，提供观察者沉浸式体验效果。VR headset: Based on Htc Vive, it receives manned/unmanned formation flight video signals and provides immersive experience for observers.
交互设备：基于捕捉设备Leap Motion进行手势捕捉，采集人的动作信息，将其转化为有人机操作指令进行虚拟有人/无人机编队仿真交互。Interaction equipment: Capture gestures based on the capture device Leap Motion, collect human motion information, and convert it into man-machine operation instructions for virtual man/drone formation simulation interaction.
脑电设备：包含信号采集子模块和信号转换子模块。基于neuracle脑电采集系统，采集用户脑电信号。基于NeuroCube单元，采用neuracle配套软件开发，转换为相应控制指令，指挥无人机，完成有人/无人编队虚拟仿真。EEG equipment: including signal acquisition sub-module and signal conversion sub-module. Based on the neuracle EEG acquisition system, the user's EEG signal is collected. Based on the NeuroCube unit, it is developed with neuracle supporting software and converted into corresponding control commands to command the UAV and complete the virtual simulation of manned/unmanned formation.
参见图2，为基于脑电设备的有人机与无人机的协同仿真方法流程图，通过主控平台启动仿真进程，提供编队任务的初始化条件，构建主从结构的实时仿真环境；仿真主控机运行主控软件，脑电设备采集模拟的有人机指令通过无线通讯方式发送控制指令到控制器，控制器经过仿真计算通过串口发送控制信号给仿真从机，仿真目标机模拟有人/无人机动力学模型和环境模型，提供有人/无人机的状态信息，与无人机控制器形成控制闭环。仿真数据回传到主控平台上，通过数据库，进行统一的管理和分析。主控计算机将仿真位置姿态角信息发送给视景VR平台，进行可视化仿真。Referring to Figure 2, it is a flow chart of the co-simulation method of man-machine and unmanned aerial vehicle based on EEG equipment. The simulation process is started through the main control platform, the initialization conditions of the formation task are provided, and the real-time simulation environment of the master-slave structure is constructed; the simulation master control The machine runs the main control software, and the EEG equipment collects and simulates man-machine commands and sends control commands to the controller through wireless communication. The mechanical model and the environment model provide the state information of manned/unmanned aerial vehicles, and form a closed control loop with the UAV controller. The simulation data is sent back to the main control platform for unified management and analysis through the database. The main control computer sends the simulation position and attitude angle information to the visual VR platform for visual simulation.
参见图3，为平台管理软件结构示意图，平台管理软件的功能是实现有人/无人机编队控制系统从控制器方案设计到离线仿真、实时仿真，最后进行仿真分析的全过程。仿真平台技术层采用多种技术支撑仿真功能的实现，利用多线程Matlab引擎技术实现对基于Matlab/Simulink开发的仿真模型以及编队控制软件的支持，同时允许预先进行离线仿真。利用Matlab针对Simulink Real Time target提供的RTI驱动程序完成仿真程序向C语言代码的自动转换，实现实时代码的下载。仿真参数解析技术用于获取C代码程序仿真参数变量与Matlab编写的仿真参数的对应关系，从而可以实时监控仿真数据信息。利用数据库技术将离线仿真数据与实时仿真数据均保存到数据库对应的数据表单中。并进行性能评估。See Figure 3, which is a schematic diagram of the platform management software structure. The function of the platform management software is to realize the whole process of manned/unmanned aerial vehicle formation control system from controller scheme design to offline simulation, real-time simulation, and finally simulation analysis. The technical layer of the simulation platform adopts a variety of technologies to support the realization of the simulation function, and uses the multi-threaded Matlab engine technology to realize the support for the simulation model and formation control software developed based on Matlab/Simulink, and allows offline simulation in advance. The RTI driver program provided by Matlab for Simulink Real Time target is used to complete the automatic conversion of the simulation program to the C language code, and realize the download of the real-time code. The simulation parameter analysis technology is used to obtain the corresponding relationship between the simulation parameter variables of the C code program and the simulation parameters written by Matlab, so that the simulation data information can be monitored in real time. Both offline simulation data and real-time simulation data are stored in the corresponding data form of the database by using database technology. and perform performance evaluations.
参见图4，为视景演示软件结构图，该软件基于Unity开发，分为三个模块，具体实施方式如下：Referring to Figure 4, it is a structural diagram of the visual demonstration software. The software is developed based on Unity and is divided into three modules. The specific implementation methods are as follows:
三维虚拟场景模块：基于SketchUp、3D max建模与开发资源站Unity AssetStore、Google3D Warehouse等资源，建立丰富的模型库、场景库，并实现多种三维特效。模型库包括多种固定翼/旋翼的有人/无人机、导弹、各种军事打击目标、建筑物等三维模型；陆地场景库包括多种地形，如山地、平原、河流、森林等；基于Unity集成的Physics物理引擎实现对目标进行打击时产生的爆炸、碎片特效，基于粒子系统Shuriken实现飞行过程中产生的尾焰、尾迹等特效。3D virtual scene module: Based on resources such as SketchUp, 3D max modeling and development resource station Unity AssetStore, Google3D Warehouse, etc., build a rich model library and scene library, and realize a variety of 3D special effects. The model library includes a variety of fixed-wing/rotor manned/unmanned aerial vehicles, missiles, various military strike targets, buildings and other 3D models; the land scene library includes a variety of terrains, such as mountains, plains, rivers, forests, etc.; based on Unity The integrated Physics physics engine realizes the special effects of explosion and debris when striking the target, and realizes special effects such as tail flames and wakes during flight based on the particle system Shuriken.
人机交互模块：Leap Motion提供了Unity版本的手势识别软件包Orion，将它集成到视景演示软件中进行二次开发。实现交互功能包括，引导提示功能、图形界面功能、仿真信息查看功能、视角切换功能、沙盘功能、大屏幕多视角分屏显示功能。Human-computer interaction module: Leap Motion provides the Unity version of the gesture recognition software package Orion, which is integrated into the visual presentation software for secondary development. The realization of interactive functions includes guidance prompt function, graphical interface function, simulation information viewing function, viewing angle switching function, sand table function, and large-screen multi-angle split-screen display function.
网络通信模块：向主控单元发送用户服务请求，接收仿真数据，驱动视景演示程序的运行。基于TCP和UDP协议，设计与主控机平台管理软件之间的应用层通信协议。基于.NET版本的Socket套接字API，开发客户端通信程序，实现对发送数据包的打包和对接收数据包的解包工作。Network communication module: send user service requests to the main control unit, receive simulation data, and drive the operation of the visual demonstration program. Based on the TCP and UDP protocols, design the application layer communication protocol with the host computer platform management software. Based on the Socket socket API of the .NET version, the client communication program is developed to realize the packaging of the sending data packet and the unpacking of the receiving data packet.
参见图5，为Simulink Real Time主控平台编程流程图，利用Matlab提供的Simulink Real Time API函数库编程实现对Simulink Real Time目标机的控制功能。首先，需要用应用程序注册函数对Simulink Real Time应用程序进行注册，建立主控平台与Simulink Real Time仿真机的通讯连接；然后，将基于Matlab开发的编队控制软件以及有人/无人机仿真模型加载到相应的Simulink Real Time仿真机中；启动实时仿真，仿真过程中可以进行仿真监控和在线调参功能；判断仿真是否结束，如果没有则进入仿真监控循环，如果仿真结束则断开宿主机与目标机的连接，完成本次仿真过程。仿真完成后，将获得的实时仿真数据保存到数据库中，供后期分析处理。Referring to Fig. 5, it is the programming flow chart of the Simulink Real Time main control platform, and realizes the control function of the Simulink Real Time target machine by using the Simulink Real Time API function library provided by Matlab. First, you need to register the Simulink Real Time application with the application registration function, and establish a communication connection between the main control platform and the Simulink Real Time simulator; then, load the formation control software developed based on Matlab and the manned/unmanned aerial vehicle simulation model Go to the corresponding Simulink Real Time simulator; start the real-time simulation, the simulation monitoring and online parameter adjustment functions can be performed during the simulation; judge whether the simulation is over, if not, enter the simulation monitoring loop, if the simulation is over, disconnect the host and the target The computer is connected to complete the simulation process. After the simulation is completed, the obtained real-time simulation data is saved in the database for later analysis and processing.
参见图6，为Simulink Real Time target串口通信流程图，Simulink Real Time仿真模型的串口通讯是通过Matlab\Simulink提供的带有FIFO缓冲区的串口通信模块、按数据字头读取串口数据模块、打包数据帧模块，解包数据帧模块组成。通过判断中断类型，是否有新数据，进入写入缓冲区；通过数据预设的字头查找，判断是否为需要数据，如果满足要求，则输出数据，否则继续检索数据字头。See Figure 6, which is the flow chart of Simulink Real Time target serial port communication. The serial port communication of Simulink Real Time simulation model is through the serial port communication module with FIFO buffer provided by Matlab\Simulink, read the serial port data module according to the data header, pack The data frame module is composed of the unpacking data frame module. By judging the interrupt type, whether there is new data, it enters the write buffer; through the pre-set header search of the data, it is judged whether it is required data, if it meets the requirements, output the data, otherwise continue to search the data header.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710799662.3ACN107643695B (en) | 2017-09-07 | 2017-09-07 | Human/unmanned aerial vehicle cluster formation VR simulation method and system based on electroencephalogram |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710799662.3ACN107643695B (en) | 2017-09-07 | 2017-09-07 | Human/unmanned aerial vehicle cluster formation VR simulation method and system based on electroencephalogram |
| Publication Number | Publication Date |
|---|---|
| CN107643695Atrue CN107643695A (en) | 2018-01-30 |
| CN107643695B CN107643695B (en) | 2020-11-20 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710799662.3AActiveCN107643695B (en) | 2017-09-07 | 2017-09-07 | Human/unmanned aerial vehicle cluster formation VR simulation method and system based on electroencephalogram |
| Country | Link |
|---|---|
| CN (1) | CN107643695B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108648558A (en)* | 2018-06-22 | 2018-10-12 | 中国人民解放军国防科技大学 | Manned and unmanned aerial vehicle collaborative drilling simulation cabin |
| CN108829245A (en)* | 2018-05-30 | 2018-11-16 | 中国人民解放军军事科学院国防科技创新研究院 | A kind of virtual sand table intersection control routine based on multi-modal brain-machine interaction technology |
| CN108845802A (en)* | 2018-05-15 | 2018-11-20 | 天津大学 | Unmanned plane cluster formation interactive simulation verifies system and implementation method |
| CN108983636A (en)* | 2018-06-20 | 2018-12-11 | 浙江大学 | Human-machine intelligence's symbiosis plateform system |
| CN109085766A (en)* | 2018-08-24 | 2018-12-25 | 赵小川 | A kind of multi-modal unmanned plane human-computer interaction manipulation semi-matter simulating system and method |
| CN109189094A (en)* | 2018-09-25 | 2019-01-11 | 中国人民解放军空军工程大学 | It is a kind of to have man-machine and multiple no-manned plane composite formation resource regulating method more |
| CN110008630A (en)* | 2019-04-18 | 2019-07-12 | 哈尔滨工程大学 | Underwater Unmanned Vehicle Cluster Simulation Platform |
| CN110221620A (en)* | 2019-05-30 | 2019-09-10 | 中国人民解放军国防科技大学 | MAS-based multi-unmanned system supervision control station |
| CN110377049A (en)* | 2019-06-29 | 2019-10-25 | 天津大学 | Unmanned plane cluster flight pattern reconfigurable control method based on brain-computer interface |
| CN110764433A (en)* | 2019-10-16 | 2020-02-07 | 中山大学 | V-REP platform-based cluster unmanned aerial vehicle system parallel simulation method |
| CN111010436A (en)* | 2019-12-11 | 2020-04-14 | 北京工业大学 | A data transmission system for unmanned aerial vehicle swarm system |
| CN111474956A (en)* | 2020-04-22 | 2020-07-31 | 中国电子科技集团公司第二十八研究所 | Simulation method of unmanned aerial vehicle electronic investigation target based on simulation platform and monitoring platform |
| CN111522256A (en)* | 2020-05-07 | 2020-08-11 | 北京神舟航天软件技术有限公司 | Distributed unmanned aerial vehicle cluster simulation method based on Docker |
| CN112394649A (en)* | 2020-11-17 | 2021-02-23 | 中国人民解放军63892部队 | Time-sharing multiplexing-based group unmanned aerial vehicle internal communication behavior control platform |
| WO2021035691A1 (en)* | 2019-08-30 | 2021-03-04 | 深圳市大疆创新科技有限公司 | Unmanned aerial vehicle simulation method and simulation apparatus, and computer-readable storage medium |
| CN112506076A (en)* | 2020-12-04 | 2021-03-16 | 杭州电子科技大学 | Unmanned aerial vehicle cluster three-dimensional space simulator construction method based on flight mission |
| CN112597594A (en)* | 2020-12-28 | 2021-04-02 | 中国人民解放军军事科学院国防科技创新研究院 | System and method for verifying real position of unmanned aerial vehicle cluster in virtual environment |
| CN112596799A (en)* | 2020-12-25 | 2021-04-02 | 中电科芜湖通用航空产业技术研究院有限公司 | Extensible universal airborne task management system |
| CN112631173A (en)* | 2020-12-11 | 2021-04-09 | 中国人民解放军国防科技大学 | Brain-controlled unmanned platform cooperative control system |
| CN112987794A (en)* | 2021-04-21 | 2021-06-18 | 南京南机智农农机科技研究院有限公司 | Flight cluster simulator |
| CN113312765A (en)* | 2021-05-21 | 2021-08-27 | 中国海洋大学 | Target-marine environment-sonar coupling characteristic comprehensive simulation system and method |
| CN113543077A (en)* | 2021-05-31 | 2021-10-22 | 中国人民解放军军事科学院国防科技创新研究院 | Virtual environment unmanned aerial vehicle group communication detection system and method |
| CN113589706A (en)* | 2021-08-02 | 2021-11-02 | 天津大学 | Helicopter trailing edge flap control virtual simulation method |
| CN114063474A (en)* | 2021-12-03 | 2022-02-18 | 北京航空航天大学 | A Semi-Physical Simulation System for UAV Swarms |
| CN114578712A (en)* | 2022-03-08 | 2022-06-03 | 北京航空航天大学 | Multifunctional underwater autonomous vehicle swarm simulation system |
| CN115237000A (en)* | 2022-06-23 | 2022-10-25 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle formation cooperative countermeasure simulation test platform and test method |
| CN115755958A (en)* | 2022-11-07 | 2023-03-07 | 西北工业大学 | Manned and unmanned aircraft cooperative formation form transformation method, system and device |
| CN116027809A (en)* | 2023-02-07 | 2023-04-28 | 中国矿业大学 | Multi-quad-rotor unmanned aerial vehicle formation control method under DoS attack |
| CN116842758A (en)* | 2023-08-28 | 2023-10-03 | 中国民航管理干部学院 | A simulation platform and method for algorithm verification of civil unmanned air traffic services |
| CN116996927A (en)* | 2023-09-28 | 2023-11-03 | 四川九洲软件有限公司 | Airborne communication reconnaissance demonstration method and system |
| CN117075638A (en)* | 2023-09-26 | 2023-11-17 | 南京航空航天大学 | A collaborative resilience control method for clustered UAVs against DoS network attacks |
| CN117075496A (en)* | 2023-09-18 | 2023-11-17 | 广东昊一航空科技有限公司 | A kind of drone array operation simulation system |
| CN117270574A (en)* | 2023-11-20 | 2023-12-22 | 中国空气动力研究与发展中心计算空气动力研究所 | Fixed wing unmanned aerial vehicle flight formation test method based on virtual target |
| CN118034628A (en)* | 2024-04-09 | 2024-05-14 | 西安中科天塔科技股份有限公司 | Split screen display method and system based on spacecraft simulation |
| CN118646773A (en)* | 2024-08-15 | 2024-09-13 | 成都天巡微小卫星科技有限责任公司 | An intelligent machine system for manned/unmanned formation and its computing resource calling method |
| CN119312230A (en)* | 2024-12-10 | 2025-01-14 | 河北军涛科技有限公司 | Aerial vehicle identification training interactive method and system based on VR |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102508439A (en)* | 2011-11-18 | 2012-06-20 | 天津大学 | HLA (High Level Architecture)-based multi-unmmaned aerial vehicle distributed simulation method |
| CN103093047A (en)* | 2013-01-12 | 2013-05-08 | 天津大学 | Typical aircraft visual simulation system |
| CN103246204A (en)* | 2013-05-02 | 2013-08-14 | 天津大学 | Simulation and authentication method and device for multi-UAV (unmanned-aerial-vehicle) system |
| CN103699106A (en)* | 2013-12-30 | 2014-04-02 | 合肥工业大学 | Multi-unmanned aerial vehicle cooperative task planning simulation system based on VR-Forces simulation platform |
| CN104615010A (en)* | 2014-12-26 | 2015-05-13 | 成都飞机工业（集团）有限责任公司 | FlightGear and VC hybrid system developing method based on network communication |
| CN106200679A (en)* | 2016-09-21 | 2016-12-07 | 中国人民解放军国防科学技术大学 | Single operation person's multiple no-manned plane mixing Active Control Method based on multi-modal natural interaction |
| US9721478B2 (en)* | 2009-12-01 | 2017-08-01 | The Boeing Company | Integrated live and simulation environment system for an aircraft |
| CN107065909A (en)* | 2017-04-18 | 2017-08-18 | 南京邮电大学 | A kind of flight control system based on BCI |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9721478B2 (en)* | 2009-12-01 | 2017-08-01 | The Boeing Company | Integrated live and simulation environment system for an aircraft |
| CN102508439A (en)* | 2011-11-18 | 2012-06-20 | 天津大学 | HLA (High Level Architecture)-based multi-unmmaned aerial vehicle distributed simulation method |
| CN103093047A (en)* | 2013-01-12 | 2013-05-08 | 天津大学 | Typical aircraft visual simulation system |
| CN103246204A (en)* | 2013-05-02 | 2013-08-14 | 天津大学 | Simulation and authentication method and device for multi-UAV (unmanned-aerial-vehicle) system |
| CN103699106A (en)* | 2013-12-30 | 2014-04-02 | 合肥工业大学 | Multi-unmanned aerial vehicle cooperative task planning simulation system based on VR-Forces simulation platform |
| CN104615010A (en)* | 2014-12-26 | 2015-05-13 | 成都飞机工业（集团）有限责任公司 | FlightGear and VC hybrid system developing method based on network communication |
| CN106200679A (en)* | 2016-09-21 | 2016-12-07 | 中国人民解放军国防科学技术大学 | Single operation person's multiple no-manned plane mixing Active Control Method based on multi-modal natural interaction |
| CN107065909A (en)* | 2017-04-18 | 2017-08-18 | 南京邮电大学 | A kind of flight control system based on BCI |
| Title |
|---|
| 李一波等: "有人/无人机编队三位可视化系统的软件设计与实现", 《智能系统学报》* |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108845802A (en)* | 2018-05-15 | 2018-11-20 | 天津大学 | Unmanned plane cluster formation interactive simulation verifies system and implementation method |
| CN108829245A (en)* | 2018-05-30 | 2018-11-16 | 中国人民解放军军事科学院国防科技创新研究院 | A kind of virtual sand table intersection control routine based on multi-modal brain-machine interaction technology |
| CN108983636B (en)* | 2018-06-20 | 2020-07-17 | 浙江大学 | Human-machine intelligent symbiosis platform system |
| CN108983636A (en)* | 2018-06-20 | 2018-12-11 | 浙江大学 | Human-machine intelligence's symbiosis plateform system |
| CN108648558A (en)* | 2018-06-22 | 2018-10-12 | 中国人民解放军国防科技大学 | Manned and unmanned aerial vehicle collaborative drilling simulation cabin |
| CN109085766A (en)* | 2018-08-24 | 2018-12-25 | 赵小川 | A kind of multi-modal unmanned plane human-computer interaction manipulation semi-matter simulating system and method |
| CN109189094A (en)* | 2018-09-25 | 2019-01-11 | 中国人民解放军空军工程大学 | It is a kind of to have man-machine and multiple no-manned plane composite formation resource regulating method more |
| CN109189094B (en)* | 2018-09-25 | 2021-08-13 | 中国人民解放军空军工程大学 | Resource scheduling method for hybrid formation of multiple human-machines and multiple unmanned aerial vehicles |
| CN110008630A (en)* | 2019-04-18 | 2019-07-12 | 哈尔滨工程大学 | Underwater Unmanned Vehicle Cluster Simulation Platform |
| CN110008630B (en)* | 2019-04-18 | 2022-07-15 | 哈尔滨工程大学 | Underwater Unmanned Vehicle Cluster Simulation Platform |
| CN110221620B (en)* | 2019-05-30 | 2022-06-07 | 中国人民解放军国防科技大学 | A MAS-based multi-unmanned system supervisory control station |
| CN110221620A (en)* | 2019-05-30 | 2019-09-10 | 中国人民解放军国防科技大学 | MAS-based multi-unmanned system supervision control station |
| CN110377049B (en)* | 2019-06-29 | 2022-05-17 | 天津大学 | Brain-computer interface-based unmanned aerial vehicle cluster formation reconfiguration control method |
| CN110377049A (en)* | 2019-06-29 | 2019-10-25 | 天津大学 | Unmanned plane cluster flight pattern reconfigurable control method based on brain-computer interface |
| WO2021035691A1 (en)* | 2019-08-30 | 2021-03-04 | 深圳市大疆创新科技有限公司 | Unmanned aerial vehicle simulation method and simulation apparatus, and computer-readable storage medium |
| CN110764433A (en)* | 2019-10-16 | 2020-02-07 | 中山大学 | V-REP platform-based cluster unmanned aerial vehicle system parallel simulation method |
| CN111010436A (en)* | 2019-12-11 | 2020-04-14 | 北京工业大学 | A data transmission system for unmanned aerial vehicle swarm system |
| CN111010436B (en)* | 2019-12-11 | 2022-05-17 | 北京工业大学 | A data transmission system for unmanned aerial vehicle swarm system |
| CN111474956B (en)* | 2020-04-22 | 2022-08-23 | 中国电子科技集团公司第二十八研究所 | Simulation method of unmanned aerial vehicle electronic investigation target based on simulation platform and monitoring platform |
| CN111474956A (en)* | 2020-04-22 | 2020-07-31 | 中国电子科技集团公司第二十八研究所 | Simulation method of unmanned aerial vehicle electronic investigation target based on simulation platform and monitoring platform |
| CN111522256A (en)* | 2020-05-07 | 2020-08-11 | 北京神舟航天软件技术有限公司 | Distributed unmanned aerial vehicle cluster simulation method based on Docker |
| CN112394649B (en)* | 2020-11-17 | 2022-08-30 | 中国人民解放军63892部队 | Time-sharing multiplexing-based group unmanned aerial vehicle internal communication behavior control platform |
| CN112394649A (en)* | 2020-11-17 | 2021-02-23 | 中国人民解放军63892部队 | Time-sharing multiplexing-based group unmanned aerial vehicle internal communication behavior control platform |
| CN112506076A (en)* | 2020-12-04 | 2021-03-16 | 杭州电子科技大学 | Unmanned aerial vehicle cluster three-dimensional space simulator construction method based on flight mission |
| CN112631173A (en)* | 2020-12-11 | 2021-04-09 | 中国人民解放军国防科技大学 | Brain-controlled unmanned platform cooperative control system |
| CN112596799A (en)* | 2020-12-25 | 2021-04-02 | 中电科芜湖通用航空产业技术研究院有限公司 | Extensible universal airborne task management system |
| CN112597594B (en)* | 2020-12-28 | 2024-04-26 | 中国人民解放军军事科学院国防科技创新研究院 | System and method for verifying real position of unmanned aerial vehicle group in virtual environment |
| CN112597594A (en)* | 2020-12-28 | 2021-04-02 | 中国人民解放军军事科学院国防科技创新研究院 | System and method for verifying real position of unmanned aerial vehicle cluster in virtual environment |
| CN112987794A (en)* | 2021-04-21 | 2021-06-18 | 南京南机智农农机科技研究院有限公司 | Flight cluster simulator |
| CN113312765A (en)* | 2021-05-21 | 2021-08-27 | 中国海洋大学 | Target-marine environment-sonar coupling characteristic comprehensive simulation system and method |
| CN113543077A (en)* | 2021-05-31 | 2021-10-22 | 中国人民解放军军事科学院国防科技创新研究院 | Virtual environment unmanned aerial vehicle group communication detection system and method |
| CN113589706A (en)* | 2021-08-02 | 2021-11-02 | 天津大学 | Helicopter trailing edge flap control virtual simulation method |
| CN114063474A (en)* | 2021-12-03 | 2022-02-18 | 北京航空航天大学 | A Semi-Physical Simulation System for UAV Swarms |
| CN114063474B (en)* | 2021-12-03 | 2023-06-06 | 北京航空航天大学 | A Simulation Method of Semi-Physical Simulation System Based on UAV Swarm |
| CN114578712A (en)* | 2022-03-08 | 2022-06-03 | 北京航空航天大学 | Multifunctional underwater autonomous vehicle swarm simulation system |
| CN114578712B (en)* | 2022-03-08 | 2023-09-26 | 北京航空航天大学 | Multifunctional underwater autonomous vehicle cluster simulation system |
| CN115237000A (en)* | 2022-06-23 | 2022-10-25 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle formation cooperative countermeasure simulation test platform and test method |
| CN115237000B (en)* | 2022-06-23 | 2023-08-04 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle formation cooperative countermeasure simulation test platform and test method |
| CN115755958A (en)* | 2022-11-07 | 2023-03-07 | 西北工业大学 | Manned and unmanned aircraft cooperative formation form transformation method, system and device |
| CN116027809B (en)* | 2023-02-07 | 2023-07-18 | 中国矿业大学 | Multi-quad-rotor unmanned aerial vehicle formation control method under DoS attack |
| CN116027809A (en)* | 2023-02-07 | 2023-04-28 | 中国矿业大学 | Multi-quad-rotor unmanned aerial vehicle formation control method under DoS attack |
| CN116842758A (en)* | 2023-08-28 | 2023-10-03 | 中国民航管理干部学院 | A simulation platform and method for algorithm verification of civil unmanned air traffic services |
| CN116842758B (en)* | 2023-08-28 | 2024-03-19 | 中国民航管理干部学院 | Simulation platform and method for civil unmanned aerial vehicle air traffic service algorithm verification |
| CN117075496A (en)* | 2023-09-18 | 2023-11-17 | 广东昊一航空科技有限公司 | A kind of drone array operation simulation system |
| CN117075638B (en)* | 2023-09-26 | 2024-02-06 | 南京航空航天大学 | Cluster unmanned aerial vehicle cooperative toughness control method for DoS network attack |
| CN117075638A (en)* | 2023-09-26 | 2023-11-17 | 南京航空航天大学 | A collaborative resilience control method for clustered UAVs against DoS network attacks |
| CN116996927A (en)* | 2023-09-28 | 2023-11-03 | 四川九洲软件有限公司 | Airborne communication reconnaissance demonstration method and system |
| CN117270574B (en)* | 2023-11-20 | 2024-01-26 | 中国空气动力研究与发展中心计算空气动力研究所 | Fixed wing unmanned aerial vehicle flight formation test method based on virtual target |
| CN117270574A (en)* | 2023-11-20 | 2023-12-22 | 中国空气动力研究与发展中心计算空气动力研究所 | Fixed wing unmanned aerial vehicle flight formation test method based on virtual target |
| CN118034628A (en)* | 2024-04-09 | 2024-05-14 | 西安中科天塔科技股份有限公司 | Split screen display method and system based on spacecraft simulation |
| CN118034628B (en)* | 2024-04-09 | 2024-06-18 | 西安中科天塔科技股份有限公司 | Split screen display method and system based on spacecraft simulation |
| CN118646773A (en)* | 2024-08-15 | 2024-09-13 | 成都天巡微小卫星科技有限责任公司 | An intelligent machine system for manned/unmanned formation and its computing resource calling method |
| CN119312230A (en)* | 2024-12-10 | 2025-01-14 | 河北军涛科技有限公司 | Aerial vehicle identification training interactive method and system based on VR |
| Publication number | Publication date |
|---|---|
| CN107643695B (en) | 2020-11-20 |
| Publication | Publication Date | Title |
|---|---|---|
| CN107643695B (en) | Human/unmanned aerial vehicle cluster formation VR simulation method and system based on electroencephalogram | |
| CN102508439B (en) | HLA (High Level Architecture)-based multi-unmmaned aerial vehicle distributed simulation method | |
| CN108845802A (en) | Unmanned plane cluster formation interactive simulation verifies system and implementation method | |
| CN103246204B (en) | Multiple no-manned plane system emulation and verification method and device | |
| CN107942720A (en) | A kind of online flight Simulation System of portable type ground | |
| CN103092212B (en) | The simulation and verification platform of micro-gastric carcinoma system and implementation method | |
| CN106020234B (en) | Method, device and device for flight control of unmanned aerial vehicle | |
| CN101398866A (en) | Aircraft vision emulation system | |
| CN111767031B (en) | A simulation-based nuclear power industry Internet experimental bed | |
| CN104615010A (en) | FlightGear and VC hybrid system developing method based on network communication | |
| CN113467275A (en) | Unmanned aerial vehicle cluster flight simulation system based on real object airborne equipment | |
| CN102426424A (en) | Visual emulation method of near-space flying vehicle based on distributed framework | |
| CN112558624B (en) | Spacecraft autonomous mission planning verification deployment integrated intelligent computing system | |
| WO2024244210A1 (en) | Attitude control simulation method and system for launch vehicle | |
| CN115859699B (en) | Large-scale fine-grained cluster ammunition simulation system for providing three-dimensional scene rendering | |
| CN104091017A (en) | Multi-object real-time data driving general situation display system and method based on STK | |
| CN119024717B (en) | A virtual-reality combined simulation system and method supporting large-scale UAV cluster flight tests | |
| CN116540568B (en) | Large-scale distributed unmanned aerial vehicle cluster simulation system | |
| CN116522570A (en) | Intelligent unmanned cluster system area coverage relay communication application simulation and test system | |
| CN105511295A (en) | STKX component-based satellite vision real-time simulation system and method thereof | |
| CN116566792B (en) | A distributed simulation method and system for multiple unmanned systems | |
| CN110634350A (en) | A Simulation System of Aircraft Simulation Based on Distributed Virtual Reality | |
| CN102122136A (en) | Simulation method for synchronously controlling a plurality of airplanes | |
| CN107194108A (en) | Intelligent transportation analogue system and emulation mode based on miniature group robot | |
| CN102184655A (en) | Portable processing platform of false or true scene signals |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |