CN112297030A - Interactive bionic robot fish system based on mixed reality - Google Patents

Abstract

The invention discloses an interactive bionic robot fish system based on mixed reality, which comprises a bionic robot fish, a wireless handle and mixed reality head-mounted display equipment, and integrates the overall design technology, the bionic control technology, the multi-sensor fusion technology, the image processing technology and the like of the bionic robot fish. Breaks through the harsh limitation of the application environment of the bionic robot fish, and saves water resources to a certain extent. The bionic robot fish acts in the air, and by combining with the reasonable arrangement of the virtual environment space, a user can control the bionic robot fish to realize various bionic actions such as advancing, turning, floating, submerging and the like under water, and a new revolution is brought to related products of the traditional robot.

Description

Interactive bionic robot fish system based on mixed reality

Technical Field

The invention relates to the technical field of bionic robots, in particular to an interactive bionic robot fish system based on mixed reality.

Background

Augmented Reality (AR) is an emerging technology developed in recent years, which fuses a virtual environment generated by a computer with a real scene around a user by means of a photoelectric technology, an interactive technology, a multi-sensing technology and a computer image and multimedia technology, so that the user can be certain from a sensory effect that the virtual environment is a component of the real scene around the user, thereby enhancing and expanding the perception capability of the user on the surrounding environment. Mixed Reality (MR) is a form between a virtual scene and a real scene, and includes augmented Reality and virtual Reality.

The bionic robotic fish is used as an integrated body of underwater high-technology instruments and equipment, and has wide application prospect and great potential value in the fields of military affairs, civilian use, scientific research and the like. At present, various underwater bionic robot fishes developed by scientific research institutions at home and abroad can simulate various motion modes of starting, turning and the like of underwater organisms, but are difficult to compare favorably with the organisms in the aspects of propulsion speed, propulsion efficiency and the like. Therefore, most of the related technical researches on the bionic robot fish at the present stage are focused on the aspects of propulsion mechanism, mechanism design and optimization, motion control, underwater bionic materials and the like.

The mixed reality technology is mainly used in a teaching system and used for guiding operating personnel to operate and learn certain equipment, and the special interest of the bionic robot fish is mainly focused on the design aspect of a body in machinery, control, performance and the like, so that the technical scheme of forming a special interaction system by combining the two is not discovered. The invention is based on the two technologies and innovatively combines the two technologies for application.

Disclosure of Invention

Based on the defects of the prior art, the invention provides an interactive bionic robot fish system based on mixed reality, and the interactive bionic robot fish system can be applied to places such as schools, amusement parks, science and technology museums and the like.

The technical scheme includes that the interactive bionic robot fish system based on mixed reality comprises a bionic robot fish, a wireless handle and mixed reality head-mounted display equipment, wherein the bionic robot fish is fixed on a desktop through a bottom end support, a mouth, a pectoral fin, a waist and a tail fin of the bionic robot fish can move, the wireless handle is used for wirelessly controlling the bionic robot fish, movable joints of the bionic robot fish can be respectively controlled through a handle rocker and a button, the mixed reality head-mounted display equipment comprises a computer host, a holographic projection system, a visual sensor and an inertial sensor, and the holographic projection system, the visual sensor and the inertial sensor are in wired connection to carry out data communication with the computer host.

Furthermore, the vision sensor collects the pose information of the bionic robot fish, transmits the pose information to the central processing system of the computer host to calculate a three-dimensional model of the bionic robot fish and the surrounding environment, generates image information, and controls the holographic projection system to project an area around the bionic robot fish through the holographic perspective mirror, wherein the area comprises virtual animation and a control menu.

Further, the wireless handle selects the virtual menu option through a handle key.

Furthermore, a power switch and an indicator light are arranged at the bottom of the bionic robot fish.

The invention has the beneficial effects that: the invention combines the bionic robot fish with the mixed reality technology, breaks the harsh limitation of the application environment of the bionic robot fish, and saves water resources to a certain extent. The bionic robot fish acts in the air, and by combining with the reasonable arrangement of the virtual environment space, a user can control the bionic robot fish to realize various bionic actions such as advancing, turning, floating, submerging and the like under water, and a new revolution is brought to related products of the traditional robot.

Drawings

FIG. 1 is a front view of the biomimetic robotic fish of the present invention.

FIG. 2 is a bottom view of the biomimetic robotic fish of the present invention.

Fig. 3 is a schematic view of a wireless handle of the present invention.

Fig. 4 is a schematic diagram illustrating the operation of the present invention.

The labels in the figure are: 1-head shell, 2-indicator light, 3-jaw, 4-occlusion mechanism, 5-right pectoral fin, 6-right pectoral fin steering engine, 7-left pectoral fin, 8-left pectoral fin steering engine, 9-energy control cabin, 10-wireless module, 11-main control board, 12-battery, 13-cabin cover, 14-waist joint support, 15-waist joint, 16-waist joint steering engine, 17-tail handle steering engine, 18-tail fin, 19-tail handle, 20-charging port, 21-power switch.

Detailed Description

The invention will now be further described with reference to fig. 1 to 4, and the techniques adopted by the invention include: the bionic robot fish comprises a bionic robot fish overall design technology, a bionic control technology, a multi-sensor fusion technology, an image processing technology and the like.

As shown in fig. 1 and 2, the biomimetic robotic fish comprises: the device comprises a head shell 1, anindicator light 2, alower jaw 3, anocclusion mechanism 4, a rightpectoral fin 5, a right pectoralfin steering engine 6, a leftpectoral fin 7, a left pectoralfin steering engine 8, anenergy control cabin 9, awireless module 10, amain control board 11, abattery 12, acabin cover 13, a waistjoint support 14, awaist joint 15, a waistjoint steering engine 16, a tailhandle steering engine 17, atail fin 18, atail handle 19, acharging port 20 and a power switch 21.

The wireless handle looks similar to a game pad on the market, and typically has two sets of joysticks, a set of cross directional buttons, and X, Y, A, B, RB, LB, RT, LT, BACK, TUBRO, SHIFT, START buttons, with a built-in wireless communication module.

The mixed reality head-mounted display equipment is integrated by a computer host, a holographic projection system, a visual sensor and an inertial sensor, wherein the holographic projection system, the visual sensor and the inertial sensor are in data communication with the computer host by adopting wired connection.

As shown in fig. 4, three of the hardware components are, 1) a mixed reality head-mounted display device, 2) a wireless handle, and 3) a biomimetic robotic fish.

Firstly, placing the bionic robot fish (including a base) on a desktop, keeping the desktop clean and tidy without sundries, preferably setting the environment background to be monochrome, and lighting an indicator light after the bionic robot fish turns on a bottom power switch;

secondly, the mixed reality head-mounted display equipment needs to be worn on the head of an operator, the vision sensor acquires the pose information of the bionic robot fish, the cameras can acquire depth maps of different angles, and then different depth maps are accumulated, so that the accurate three-dimensional models of the bionic robot fish and the surrounding environment are calculated by the central processing system of the computer host by means of technologies such as stereoscopic vision and the like;

after acquiring a three-dimensional scene model of the bionic robot fish and the surrounding, generating image information by a computer central processing system, controlling a holographic projection system to project an area around the bionic robot fish through a holographic perspective mirror, wherein the content comprises information such as virtual animation, a control menu and the like, and an inertial sensor can acquire pose information in real time;

and fourthly, after the projection area is successfully matched with the bionic robot fish, an operator can operate the bionic robot fish and the mixed reality head-mounted display equipment through the wireless handle, and the wireless handle can realize the motion control of theindicator light 2, thelower jaw 3, the rightpectoral fin 5, the leftpectoral fin 7, thewaist joint 15 and thecaudal peduncle 19 of the bionic robot fish. The wireless handle may also select virtual menu options via handle buttons.

The bionic robot fish simulates an underwater real organism, namely a finless porpoise, has the size of 0.5m, can move at the mouth, pectoral fins, waist and tail fins, and can be fixed on a table top through a bottom support. The wireless handle can realize the wireless control to the bionic robot fish, and each movable joint of the bionic robot fish can be respectively controlled through the handle rocker and the key. The mixed reality head-mounted display device may provide a visualized simulated environment in which the biomimetic robotic fish and the simulated object coexist and interact in real-time. The operator can know information such as the internal structure, the component parts, the loading and unloading process and the like of the bionic robot fish through the simulated environment, and controls the bionic robot fish to move in the simulated environment through the wireless handle, so that the bionic robot fish interacts with a simulated object in real time, and the interactivity, the science and education performance and the entertainment performance are improved in a small use space. The invention can be applied to places such as schools, amusement parks, science and technology museums and the like.

Claims (4)

1. Interactive bionic robot fish system based on mixed reality, its characterized in that: including bionical machine fish, wireless handle and mixed reality wear-type display device, bionical machine fish be fixed in the desktop through the bottom sprag, mouth, pectoral fin, waist, the equal motion of tail fin of bionical machine fish, wireless handle be used for the wireless control to bionical machine fish, can control the movable joint of bionical machine fish respectively through handle rocker and button, mixed reality wear-type display device include computer host computer, holographic projection system, visual sensor, inertial sensor, holographic projection system, visual sensor and computer host computer adopt wired connection to carry out data communication.

2. The mixed reality based interactive biomimetic robotic fish system as recited in claim 1, wherein: the vision sensor collects the pose information of the bionic robot fish, transmits the pose information to the central processing system of the computer host to calculate a three-dimensional model of the bionic robot fish and the surrounding environment, generates image information, and controls the holographic projection system to project an area around the bionic robot fish through the holographic perspective lens, wherein the area comprises virtual animations and a control menu.

3. The mixed reality based interactive biomimetic robotic fish system as recited in claim 1, wherein: the wireless handle selects the virtual menu option through a handle key.

4. The mixed reality based interactive biomimetic robotic fish system as recited in claim 1, wherein: and a power switch and an indicator light are arranged at the bottom of the bionic robot fish.

Images