Disclosure of Invention
Aiming at the problems, the invention provides a coordinate-based intra-operative navigation image registration method, which constructs a coordinate conversion relation in real time in an operation and performs space coordinate conversion, avoids the problem that special conditions in the operation are not matched with a preoperative space conversion mapping function, shortens or removes conversion time, corrects the construction of a coordinate system, and solves the error problem, and the method specifically comprises the following steps:
s1, acquiring an image of a register through an image system to obtain a coordinate of an optical small ball on the register in an image coordinate system, wherein the register comprises a first plane and a second plane which are not parallel, the first plane is provided with N optical small balls connected through a matching structure, the optical small balls are separated from the first plane, the second plane is provided with N steel balls, the image system acquires N coordinate points by identifying the N steel balls and the N optical small balls on the second plane of the register, N is a natural number greater than or equal to 3, and the coordinate points of the optical small balls are corrected based on the distance to obtain the coordinate points of the corrected optical small balls on the first plane;
s2, an optical tracking system tracks the coordinates of the optical small ball on the register in a three-dimensional space;
and S3, completing registration based on the coordinates of the optical small ball in the image coordinate system and the coordinates of the optical small ball in the three-dimensional space.
The correction is to calculate the relative displacement of the coordinate points according to the relative distance.
The registration device is replaced by a registration device which comprises N optical pellets on each of two planes, the optical pellets are connected to the planes through a matching structure, distances between the optical pellets and the planes exist, the image system obtains N optical pellet coordinates by identifying the N optical pellets on the two planes, the coordinates of the N optical pellets are corrected to obtain corrected optical pellet coordinates, and an image coordinate system is built based on the corrected optical pellet coordinates.
The registration device is replaced by N steel balls and N optical small balls on a first plane, N steel balls are arranged on a second plane, the steel balls on the first plane are connected with the optical small balls through a first matching structure, the steel balls are connected on the first plane through a second matching structure, the optical small balls, the first matching structure, the steel balls and the second matching structure are located on the same axis, the axis is perpendicular to the first plane, the steel balls are separated from the first plane, and the image system obtains N coordinate points of the registration device by identifying the N steel balls or the N optical small balls on the registration device;
The construction of the image coordinate system comprises the steps of obtaining an image of a register, extracting steel ball coordinates on a first plane and steel ball coordinates on a second plane in the image, correcting the coordinates of the steel balls on the first plane to obtain corrected steel ball coordinates on the first plane, and constructing a three-dimensional coordinate system based on the corrected steel ball coordinates on the first plane and the corrected steel balls on the second plane.
S3, replacing the coordinate of the steel ball of the first plane in the image coordinate system with the coordinate after the steel ball calculation is obtained after the relative displacement calculation is carried out on the basis of the relative distance between the steel ball and the optical small ball, and completing registration on the basis of the coordinate after the steel ball calculation and the coordinate of the optical small ball in the three-dimensional space.
The registration device is replaced by the device which comprises two planes, the two planes are not parallel, N optical pellets are arranged on each plane, and the image system obtains coordinates in the image coordinate system of the N optical pellets by identifying the N optical pellets on any one plane.
The construction of the image coordinate system comprises the steps of obtaining an image of a register, extracting coordinates of an optical ball in the image, and constructing the three-dimensional coordinate system to obtain the image coordinate system.
The register comprises a first plane and a second plane, the first plane is not parallel to the second plane, N optical pellets are arranged on the first plane, N steel balls are arranged on the second plane, and the image system obtains N optical pellet coordinates by identifying the N optical pellets on the first plane.
The construction of the image coordinate system comprises the steps of obtaining an image of a register, extracting coordinates of an optical ball in the image and coordinates of a steel column, and constructing the three-dimensional coordinate system to obtain the image coordinate system.
The method further comprises the steps of registering the preoperative image, acquiring a preoperative three-dimensional image through the image system, mapping the coordinate relationship between the preoperative three-dimensional image and the intraoperative two-dimensional image acquired by the image system to obtain a preoperative coordinate relationship, and registering the preoperative three-dimensional image and the optical tracking system based on the registering of the image system and the optical tracking system and the preoperative coordinate relationship.
It is an object of the present invention to provide a computer program product comprising a computer program or instructions thereon for execution by a processor to perform a method of coordinate-based intra-operative navigation image registration as described above.
It is an object of the present invention to provide a computer device comprising a memory, a processor and a computer program or instructions stored on the memory, which are executed by the processor to perform the method of coordinate-based intra-operative navigation image registration described above.
It is an object of the present invention to provide a computer readable storage medium having stored thereon a computer program or instructions for execution by a processor to perform a method of coordinate-based intra-operative navigation image registration as described above.
The invention has the advantages that:
1 the coordinates of the image system and the optical tracking system are unified through image registration of navigation in operation, namely, the coordinates of the image system calculate the coordinates of the optical tracking system through displacement, so that the conversion steps of the coordinates between the systems are saved, the data processing time of the system is shortened, the real-time requirement of navigation in operation is met, and the calculation efficiency is improved.
2, By carrying out simple displacement calculation on the optical small ball on the register as a coordinate, the error in the construction of an image coordinate system is solved, the accuracy of recognition sites is improved, the accurate presentation of the position of the instrument or the patient in the operation navigation is facilitated, and meanwhile, the consumable is saved.
Detailed Description
In order to enable those skilled in the art to better understand the present invention, the following description will make clear and complete descriptions of the technical solutions according to the embodiments of the present invention with reference to the accompanying drawings.
In some of the flows described in the specification and claims of the present invention and in the above figures, a plurality of operations appearing in a particular order are included, but it should be clearly understood that the operations may be performed in other than the order in which they appear herein or in parallel, the sequence numbers of the operations such as S101, S102, etc. are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types.
Fig. 1 is a schematic diagram of a method for registration of intra-operative navigation images based on coordinates according to an embodiment of the present invention, which specifically includes:
The method comprises the steps of S1, obtaining the coordinates of optical pellets on a register in an image coordinate system by an image system, wherein the register comprises a first plane and a second plane which are not parallel, the first plane is provided with N optical pellets connected through a matching structure, the optical pellets are separated from the first plane by N steel balls, the second plane is provided with N steel balls, the image system obtains N coordinate points by identifying N steel balls and N optical pellets on the second plane of the register, N is a natural number greater than or equal to 3, and correcting the coordinate points of the optical pellets based on the distance to obtain the corrected coordinate points of the optical pellets on the first plane;
in one embodiment, the imaging system is a C-arm fluoroscope.
In one embodiment, the correction is a relative displacement calculation of the coordinate points based on the relative distances.
In one embodiment, the register comprises two planes, the two planes are not parallel, each plane has N optical pellets, and the imaging system obtains N optical pellet coordinates by identifying the N optical pellets on any one plane.
The construction of the image coordinate system comprises the steps of obtaining an image of a register, extracting coordinates of an optical ball in the image, and constructing the three-dimensional coordinate system to obtain the image coordinate system.
In another embodiment, the register comprises a first plane and a second plane, the first plane is not parallel to the second plane, the first plane is provided with N optical pellets, the second plane is provided with N steel balls, and the image system obtains N optical pellet coordinates by identifying the N optical pellets on the first plane.
The construction of the image coordinate system comprises the steps of obtaining an image of a register, extracting coordinates of an optical ball in the image and coordinates of a steel column, and constructing the three-dimensional coordinate system to obtain the image coordinate system.
In another embodiment, the register is replaced by a register comprising two planes, each plane is provided with N optical pellets, the optical pellets are connected to the planes through a matching structure, the optical pellets are separated from the planes by a distance, the image system obtains N optical pellet coordinates by identifying the N optical pellets on the two planes, the coordinates of the N optical pellets are corrected to obtain corrected optical pellet coordinates, and an image coordinate system is constructed based on the corrected optical pellet coordinates.
In another embodiment, the register is replaced by N steel balls and N optical balls on a first plane, the N steel balls are arranged on a second plane, the steel balls on the first plane are connected with the optical balls through a first matching structure, the steel balls are connected on the first plane through a second matching structure, the optical balls, the first matching structure, the steel balls and the matching structure are located on the same axis, the axis is perpendicular to the first plane, a distance exists between the steel balls and the first plane, and the image system obtains N coordinate points of the register by identifying the N steel balls or the N optical balls on the register;
The construction of the image coordinate system comprises the steps of obtaining an image of a register, extracting steel ball coordinates on a first plane and steel ball coordinates on a second plane in the image, correcting the coordinates of the steel balls on the first plane to obtain corrected steel ball coordinates on the first plane, and constructing a three-dimensional coordinate system based on the corrected steel ball coordinates on the first plane and the corrected steel balls on the second plane.
In another embodiment, the register is replaced by N steel balls and N optical balls on a first plane, the N steel balls are arranged on a second plane, the steel balls on the first plane are connected with the optical balls through a first matching structure, the steel balls are connected on the first plane through a second matching structure, the optical balls, the first matching structure, the steel balls and the matching structure are located on the same axis, the axis is perpendicular to the first plane, a distance exists between the steel balls and the first plane, and the image system obtains N coordinate points of the register by identifying the N steel balls or the N optical balls on the register;
The construction of the image coordinate system comprises the steps of obtaining an image of a register, extracting steel ball coordinates on a first plane and steel ball coordinates on a second plane in the image, correcting the coordinates of the steel balls on the first plane to obtain corrected steel ball coordinates on the first plane, and constructing a three-dimensional coordinate system based on the corrected steel ball coordinates on the first plane and the corrected steel balls on the second plane. And S3, replacing the coordinate of the steel ball on the first plane in the image coordinate system with the coordinate after the steel ball calculation by performing relative displacement calculation based on the relative distance between the steel ball and the optical small ball, and completing registration based on the coordinate after the steel ball calculation and the coordinate of the optical small ball in the three-dimensional space.
In a specific embodiment, as shown in fig. 4, the first structure of the register comprises a reference frame, the reference frame comprises a first bearing platform and a second bearing platform, the first bearing platform and the second bearing platform are both bearing platforms, the first bearing platform and the optical small balls are combined through a matching structure, steel balls are arranged in the middle of the matching structure, each optical small ball corresponds to one steel ball through the matching structure, the center distance between the optical small ball on each matching structure and the steel ball is fixed, the longitudinal axis of the matching structure is a straight line, the optical small ball and the steel ball are arranged along the axis of the matching structure, as shown in fig. 5, the arrangement is mainly used for the situation that when in clinical spine surgery navigation, the lateral shooting is due to large body thickness, and the fact that the coordinate system of an image system is not accurately calibrated due to unclear shooting of the optical small racket is caused by large similarity of the body density is generated, and the normal shooting can increase the accuracy of the coordinate system calibration of the image system by using the scheme. Through the arrangement, the registration ball (the optical ball and the steel ball) can be arranged in the same space through the matching structure, the space requirement cannot be increased, and because the optical ball and the steel ball are fixed distances on the same axis, the improvement cannot cause great pressure on a calculation program. The distance between the optical ball and the steel ball is 0.8-1.5cm, and the distance between the steel ball and the first plane exists.
The ball bearing device comprises a bearing table, a ball bearing, an optical small ball and a ball center connecting line, wherein the ball bearing table is arranged on the bearing table, the ball bearing can be connected with the ball bearing table along an assembling direction I, the optical small ball is connected with the ball bearing table, and the ball center connecting line of the optical small ball and the ball bearing is parallel to the assembling direction I.
The reference frame is also provided with a connecting part which is respectively connected with the first bearing table and the second bearing table, and a connecting channel is arranged through the connecting part. The connecting channel can be connected with a connecting piece, one tail end of the connecting piece is inserted into the connecting channel, and the other tail end of the connecting piece is exposed out of the connecting part. The user can grasp the connecting piece by hand or grasp the connecting piece by adopting the mechanical arm, and can adjust the space position and the posture of the registration instrument by driving the connecting piece to move, the operation mode can avoid the limbs of the user from entering the irradiation range of the electronic computer tomography apparatus or the optical registration apparatus for capturing the image, thereby being capable of reducing the radiation suffered by the user during operation and avoiding the shielding of the limbs of the user on the steel ball and the optical small ball.
Further, the connecting channel at least comprises two sections of round holes with different inner diameters so as to assemble connecting pieces with different shapes.
Further, the matching structure comprises a second matching structure connected with the steel balls, the first bearing table is provided with a matching groove, and the second matching structure is matched with the matching groove.
Further, the second matching structure is columnar, and the axis is coincident with the center of the steel ball.
Further, the matching structure also comprises a first matching structure connected with the steel ball, the first matching structure is connected with the optical small ball.
Further, the first matching structure is columnar, wherein the axis coincides with the center of the steel ball and the center of the optical ball, and the central axis of the second matching structure coincides with or does not coincide with the central axis of the first matching structure.
Further, the connecting line of the centers of any two steel balls is not coincident with the centers of other steel balls.
Further, the outer diameter of the steel ball is different from the outer diameter of the optical small ball, and specifically, the outer diameter of the optical small ball is larger than the outer diameter of the steel ball.
In a specific embodiment, as shown in fig. 6, the second structure of the register is at least provided with 3 optical balls, the 3 optical balls are connected with a first plane through a matching structure, the centers of the 3 optical balls are located on the same plane, the plane formed by the 3 optical balls is parallel to the first plane and has a distance, and the balls are at least provided with 3 balls, the centers of the 3 balls are located on a second plane, and the first plane is not parallel to the second plane;
the registration device comprises a reference frame, wherein the reference frame comprises a first bearing table and a second bearing table, the optical small ball is directly arranged on the first bearing table or is arranged on the first bearing table through a connecting rod, and the steel ball is arranged on the second bearing table through the connecting rod.
All or part of the first bearing table and the second bearing table are mirror images, and the first bearing table and the second bearing table are preferably identical in structure and mirror images, and all the small balls are framed in one space through the arrangement.
Furthermore, the first bearing platform and the second bearing platform are both bearing platforms, the first bearing platform and the first registration ball are combined through a matching structure, and the second registration ball is embedded into the second bearing platform.
The optical small balls are arranged at the edge position of the first bearing table to enlarge the distance of each small ball so as to ensure that the optical small balls can be accurately identified by the navigation system, the steel balls are distributed at the middle position of the second bearing table, the steel balls can be ensured not to interfere with each other when being identified through the arrangement in the middle, the accuracy of the identification of the image system can be ensured, an arc-shaped notch is arranged at one edge of the bearing table, an optical small ball is arranged in the middle of the notch, the length of one edge of the bearing table can be increased through the arc-shaped arrangement, and the fact that the optical small ball can be identified by the navigation system is ensured to be arranged in the middle of the notch. Because the image system identification requires the optical small ball to be close to ensure the accuracy of identification, and the navigation system identification requires the optical small ball to be far away from the optical small ball, the minimum volume of the whole instrument can be ensured under the condition that the two system identification accuracy can be met to the greatest extent through the arrangement.
In one embodiment, the method further comprises image calibration, wherein the image system is used for image calibration after constructing an image coordinate system to obtain a calibrated image coordinate system, and the image of the register is obtained to obtain the coordinates of the register under the calibrated image coordinate system.
In one embodiment, the image calibration is completed by a calibrator with N steel balls, N is a natural number greater than or equal to 3, and the image system obtains the spatial position relationship between the surgical light source and the image coordinate system by the N steel balls in the calibrator.
In one embodiment, the calibration can be completed according to Zhang Zhengyou calibration method, as long as the positions of more than 3 planes in three-dimensional space, each plane being more than 4 points, and the corresponding projection coordinates of the positions in two-dimensional images are obtained, the more planes, the more points on the planes, and the more robustness. The invention is based on the fact that as many planes as possible are constructed, each with a plurality of points. According to the present whole column, 4 straight lines (3 points on each straight line) can be formed by 12 points, and the 4 straight lines are arbitrarily taken to be two on one plane, so that 6 combinations of 2 are taken by 4, namely 6 planes, and 6 points on each plane (each plane is formed by 2 lines, and 3 points are formed on each line) meet the requirement of the Zhang Zhengyou calibration method.
In one embodiment, the method further includes image distortion correction, the image coordinate system of the image system obtains a corrected image coordinate system through image distortion correction, the image of the register is obtained based on the corrected image coordinate system, and the coordinates of the register under the corrected image coordinate system are obtained based on the image of the register.
In one embodiment, the method comprises the steps of firstly correcting image distortion of an image system, then calibrating the image system to obtain a corrected and calibrated image coordinate system, acquiring an image of a register based on the corrected and calibrated image coordinate system, and obtaining coordinates of the register under the corrected and calibrated image coordinate system based on the image of the register.
S2, an optical tracking system tracks the coordinates of the optical small ball on the register in a three-dimensional space;
in one embodiment, the optical tracking system is an NDI polar Vega and Polaris Vicra cube optical measurement, and uses near Infrared (IR) light to wirelessly detect and track navigation markers on OEM surgical instruments. They are suitable for large measuring volumes with excellent tracking accuracy and precision.
And S3, completing registration based on the coordinates of the optical pellets in the image coordinate system and the coordinates of the optical pellets in the three-dimensional space.
In one embodiment, by assembling different registration devices and adopting different registration methods, when the registration devices comprise any two planes, and each plane is N optical pellets, an image system constructs an image coordinate system by identifying the space positions of the optical pellets of the two planes and obtaining the coordinates of the optical pellets of the two planes, and an optical tracking system identifies the optical pellets of any one plane to obtain the three-dimensional space coordinates of the optical pellets, and registers the image coordinate system and the three-dimensional space based on the coordinates of the optical pellets on the registration devices in the image coordinate system and the coordinates in the three-dimensional space;
Further, after recognizing the coordinates of the optical pellets on the two planes, correcting the coordinates of the optical pellets according to the relative distance between the optical pellets and the plane where the optical pellets are located to obtain corrected coordinates of the optical pellets, and constructing an image coordinate system based on the corrected coordinates of the optical pellets.
In one embodiment, when the register comprises a first plane and a second plane, the first plane is not parallel to the second plane, N optical pellets are arranged on the first plane, N steel balls are arranged on the second plane, the image system constructs an image coordinate system by identifying the space positions of the optical pellets and the steel balls of the planes and obtaining the coordinates of the optical pellets and the coordinates of the steel balls, the optical tracking system identifies the optical pellets of the first plane to obtain the three-dimensional space coordinates of the optical pellets, the image coordinate system and the three-dimensional space are registered based on the coordinates of the optical pellets on the register in the image coordinate system and the coordinates of the three-dimensional space, the image system corrects the coordinates of the optical pellets based on the relative distance between the optical pellets and the plane after identifying the optical pellets of the plane, and constructs the image coordinate system by the corrected space positions of the optical pellets and the steel balls.
In one embodiment, when the register comprises a first plane and a second plane, the first plane is not parallel to the second plane, the first plane is provided with N steel balls and N optical balls, the second plane is provided with N steel balls, the steel balls on the first plane and the optical balls are positioned on the same axis, the axis is perpendicular to the first plane, the image system performs three-dimensional coordinate system construction by identifying the space positions of the first plane optical balls and the second plane steel balls or performs three-dimensional coordinate system construction (image coordinate) by identifying the space positions of the first plane steel balls and the second plane steel balls, coordinates of the first plane steel balls and the optical balls are obtained, the optical tracking system identifies the first plane optical balls to obtain three-dimensional space coordinates of the optical balls, the image coordinate system and the three-dimensional space are registered based on the coordinates of the optical balls on the register in the image coordinate system and the three-dimensional space, or the optical tracking system identifies the first plane optical balls to obtain three-dimensional space coordinates of the optical balls, and the coordinates of the first plane steel balls and the optical balls in the three-dimensional space are registered by relative displacement of the first plane coordinate system and the optical balls in the three-dimensional space. In another embodiment, the coordinates of the steel ball of the first plane in the image coordinate system and the coordinates of the optical small ball in the three-dimensional space are calculated according to the fixed positions to complete registration.
In one embodiment, the structure of the optical ball and the steel ball in one assembly direction is arranged on the bearing table and is positioned on the same axis, and the axis is parallel to the assembly direction, so that the image system can identify the coordinates of the same point as the steel ball, the optical tracking system can identify the optical ball or the optical ball, and the optical tracking system can identify the optical ball, namely, the same coordinates are arranged in different systems, and further, in the image registration process of the intra-operative navigation, the relation conversion between the coordinates is omitted between the image system and the optical tracking system, and the coordinates under the coordinate system of one system can be obtained by obtaining the coordinates under the coordinate system of the other system, thereby shortening the registration time of the navigation system, improving the working efficiency of the system, and meeting the requirements of the intra-operative navigation instantaneity and the calculation efficiency.
In one embodiment, the method further comprises pre-operation image registration, wherein the pre-operation three-dimensional image is acquired through the image system, the pre-operation three-dimensional image and the intra-operation two-dimensional image acquired through the image system are subjected to coordinate relation mapping to obtain a pre-operation coordinate relation, and registration of the pre-operation three-dimensional image and the optical tracking system is obtained based on registration of the image system and the optical tracking system and the pre-operation coordinate relation.
The disclosed embodiments also provide a computer program product or system comprising a computer program which, when executed by a processor, performs the above-described method steps of coordinate-based intra-operative navigation image registration.
Fig. 2 is a schematic diagram of a system for registration of intra-operative navigation images based on coordinates according to an embodiment of the present invention, which specifically includes:
the coordinate module is used for obtaining the coordinates of the optical pellets on the register in an image coordinate system by an image system, wherein the register comprises a first plane and a second plane which are not parallel, the first plane is provided with N optical pellets connected through a matching structure, the optical pellets are separated from the first plane by N steel balls, the second plane is provided with N steel balls, the image system is used for obtaining N coordinate points by identifying the N steel balls and the N optical pellets on the second plane of the register, N is a natural number greater than or equal to 3, and the coordinate points of the optical pellets are corrected based on the distance to obtain the coordinate points of the corrected optical pellets on the first plane;
The transmission module is used for tracking the coordinates of the optical small ball on the register in a three-dimensional space by the optical tracking system;
And the registration module is used for completing registration based on the coordinates of the optical small ball in the image coordinate system and the coordinates of the optical small ball in the three-dimensional space.
Fig. 3 is a schematic diagram of an apparatus for registration of intra-operative navigation images based on coordinates according to an embodiment of the present invention, which specifically includes:
The system comprises a memory and a processor, wherein the memory is used for storing program instructions, and the processor is used for calling the program instructions, and when the program instructions are executed, any one of the coordinate-based intra-operative navigation image registration method is carried out.
The disclosed embodiments also provide a computer readable storage medium storing a computer program which, when executed by a processor, performs any one of the above-described methods of coordinate-based intra-operative navigation image registration.
The results of the verification of the present verification embodiment show that assigning an inherent weight to an indication may improve the performance of the method relative to the default setting. It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form. The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units. Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program for instructing related hardware, and the program may be stored in a computer readable storage medium, where the storage medium may include a Read Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, etc.
It will be appreciated by those skilled in the art that all or part of the steps in the method of the above embodiment may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, where the medium may be a rom, a magnetic disk, or an optical disk, etc.
While the foregoing describes a computer device provided by the present invention in detail, those skilled in the art will appreciate that the foregoing description is not meant to limit the invention thereto, as long as the scope of the invention is defined by the claims appended hereto.