Disclosure of Invention
In order to achieve the above purpose, the method for identifying the deflection degree of the water jet blade comprises the following steps of S1, presetting an image template, namely taking an angle formed by a corresponding sagittal plane of the water jet blade in an ultrasonic image and an actual plane of the water jet blade as a deflection angle, taking a distance of the jet blade relative to the sagittal plane, which is caused by the deflection angle of the actual plane of the water jet blade relative to the corresponding sagittal plane of the water jet blade in the ultrasonic image, as an offset distance, acquiring preset water jet blade images corresponding to the water jet blade on the sagittal plane under a plurality of deflection angles in advance to form a preset water jet blade image template set, and/or acquiring preset jet image corresponding to the jet blade on the sagittal plane under a plurality of offset distances in advance to form a preset jet blade image template set, wherein the vertical distance from the center line of the jet blade to the sagittal plane is the offset distance;
S2, an image matching step, namely acquiring an actual water jet image corresponding to the water jet on the sagittal plane, matching the actual water jet image with an image template in the preset water jet image template set, and/or acquiring an actual jet image corresponding to the jet on the sagittal plane, and matching the actual jet image with an image template in the preset jet image template set;
when the matching result meets the preset first condition, the deflection degree of the water knife is judged to be acceptable without adjusting the pose of the water knife, and when the matching result does not meet the preset first condition, the deflection degree of the water knife is judged to be unacceptable and the pose of the water knife is judged to be required to be adjusted.
According to the technical scheme, whether the water jet knife deflects or not and whether the deflection degree is acceptable or not are preliminarily judged through image matching, so that judgment accuracy and adjustment efficiency are improved.
In a preferred mode, the rough deflection angle of the water jet is obtained based on the matching result of the actual water jet image and the preset water jet image template set, and/or the rough offset distance of the jet is obtained based on the matching result of the actual jet image and the preset jet image template set;
The first condition is that the rough deflection angle is smaller than a preset first deflection angle threshold value, and/or the rough deflection distance is smaller than a preset first deflection distance threshold value.
According to the invention, under the condition that the rough deflection angle and the rough deflection distance meet the conditions, the posture of the water knife is judged not to need to be adjusted, and misoperation can be effectively avoided.
In a preferred manner, when the matching result does not satisfy the first condition, the deflection angle calculation step is entered:
And under the condition that the matching result meets a preset second condition, acquiring the deflection angle of the water jet cutter based on the rough deflection angle and/or the rough offset distance.
In a preferred manner, if the matching result does not satisfy the second condition, an ultrasound image of at least one cross section perpendicular to the sagittal plane is acquired, and the angle of deflection of the water jet is determined based at least on the position of the imaging point of the water jet in the ultrasound image of the at least one cross section.
According to the technical scheme, under the condition that the rough deflection angle of the water knife and the rough deflection distance of the jet flow are enough to accurately judge the deflection angle of the water knife, the deflection angle of the water knife can be calculated only through the sagittal image. The deflection angle of the water jet is calculated by introducing a cross-sectional image in the case that the rough deflection angle of the water jet and the rough offset distance of the jet are insufficient to accurately determine the deflection angle of the water jet.
In a preferred mode, the second condition is that the rough deflection angle is smaller than a preset second deflection angle threshold value and/or the rough deflection distance is smaller than a preset second deflection distance threshold value, or the second condition is that the deflection degree of the water jet obtained by weighting the rough deflection angle and the rough deflection distance is smaller than a preset deflection degree threshold value.
In a preferred mode, in the deflection angle calculating step, a mapping relationship between the preset water jet image template set and the preset jet image template set is pre-established;
And judging whether the actual water jet image and the actual jet image accord with the mapping relation or not under the condition that the matching result meets a preset second condition, if so, judging that the rough deflection angle is high in credibility, and if not, judging that the rough deflection angle is low in credibility.
According to the present invention, when the deflection angle is determined to be highly reliable, the number of times of adjustment of the water jet blade can be reduced, and the burden on the patient can be reduced. Under the condition that the deflection angle is judged to be low in reliability, the correct water knife pose can be approximated step by step, and the adjustment accuracy is ensured.
In a preferred manner, in the deflection angle calculation step, in the case where the matching result does not satisfy the second condition,
If the matching result meets a preset third condition, acquiring ultrasonic images of a first cross section and a second cross section which are arranged at intervals, wherein the water jet knife is respectively presented as a first imaging point and a second imaging point in the ultrasonic images corresponding to the first cross section and the second cross section;
If the matching result does not meet the third condition, only acquiring an ultrasonic image of a cross section, and determining a corresponding linear equation of the water jet in a physical space coordinate system based on the position of an imaging point of the water jet in the ultrasonic image corresponding to the cross section and the position of an imaging point of the jet hole in the sagittal plane image;
The third condition is that the rough deflection angle is larger than a preset third deflection angle threshold value and/or the rough deflection distance is larger than a preset third deflection distance threshold value.
According to the technical scheme, when the deflection angle is relatively large, the two cross sections are introduced, so that the angle judgment can be more accurate.
In a preferred mode, in the deflection angle calculation step, when the matching result does not satisfy the second condition and satisfies the third condition, a position corresponding to the knife hole in the sagittal image is selected to acquire a cross-sectional image as the image of the first cross section.
In a preferred mode, based on the calibration relation between the ultrasonic image coordinate system and the physical space coordinate system, the first pixel coordinate and the second pixel coordinate of the first imaging point and the second imaging point in the ultrasonic image coordinate system are respectively obtained, and then the first pixel coordinate and the second pixel coordinate are converted into the first space coordinate and the second space coordinate in the physical space coordinate system.
In a preferred mode, in the deflection angle calculation step, when the rough deflection angle is greater than or equal to a preset deflection angle threshold value and the rough deflection distance is smaller than or equal to a preset deflection distance threshold value, only one cross section is arranged in an ultrasonic image, and a corresponding linear equation of the water jet in a physical space coordinate system is determined based on the position of an imaging point of the water jet on the cross section and the position of an imaging point of the knife hole on the sagittal plane.
According to the technical scheme, when the deflection angle is relatively not large, the cross section is introduced, so that the deflection angle can be accurately judged, the moving times and the distance of the ultrasonic probe are reduced, and the damage to the cavity tissue is reduced.
In a preferred mode, if the safety precision is m and the corresponding maximum deflection angle in the preset water jet image template set is θ0, in an ultrasonic image coordinate system, a distance between the second cross section and the first cross section is m×tan θ0.
In a preferred manner, the design calculation accuracy is n and the safety factor is p, based on at least errors in the ultrasound image acquisition and image recognition process, and the safety accuracy m=n×p.
In addition, the deflection degree identification system of the water knife is characterized by comprising an image acquisition module, an image template matching module and a water knife deflection degree calculation module, wherein the image acquisition module is used for acquiring ultrasonic images of the water knife and jet flow of the water knife, the image template matching module is used for matching the ultrasonic images of the water knife and jet flow of the water knife acquired by the image acquisition module with a preset water knife image template set and a preset jet flow image template set, and the water knife deflection degree calculation module is used for determining the actual deflection degree of the water knife according to the deflection degree identification method of the water knife.
Detailed Description
Various exemplary embodiments of the present application are described in detail below with reference to the accompanying drawings. The description of the exemplary embodiments is merely illustrative, and is in no way intended to limit the application, its application, or uses. The present application may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art. It should be noted that the relative arrangement of parts and steps, numerical expressions and numerical values, etc. set forth in these embodiments are to be construed as illustrative only and not limiting unless otherwise indicated.
As used herein, the word "comprising" or "comprises" and the like means that elements preceding the word encompass the elements recited after the word, and not exclude the possibility of also encompassing other elements.
All terms (including technical or scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs, unless specifically defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Parameters of, and interrelationships between, components, and control circuitry for, components, specific models of components, etc., which are not described in detail in this section, can be considered as techniques, methods, and apparatus known to one of ordinary skill in the relevant art, but are considered as part of the specification where appropriate.
It should be noted that while the operations of the method of the present application are described in a particular order, this does not require or imply that the operations must be performed in the particular order or that all of the illustrated operations be performed in order to achieve desirable results. Rather, the steps depicted in the present application may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.
Water jet deflection
The deflection of the water jet blade 20 will be described below with reference to fig. 1. Fig. 1 is a schematic diagram of the deflection of the water jet blade 20.
The X-axis, Y-axis and Z-axis shown in FIG. 1 are physical space coordinate systems. During surgery, the water knife 20 and the ultrasonic probe are arranged generally in a direction parallel to the Y-axis. For example, in a prostate-related surgical scenario, where the ultrasound probe is typically inserted transrectally into a body cavity, the water jet 20 may alternatively be transurethral or perineal into tissue, both comprising elongate shaft bodies disposed in parallel orientation to each other, facilitating visualization of the performance of the water jet 20 as a surgical device by the ultrasound probe. In particular operations, the ultrasonic probe is typically first inserted into a body cavity, and during the subsequent placement of the water blade 20, the goal is to place the water blade 20 in alignment with the ultrasonic probe, ideally where such alignment is desired to be strictly parallel to the axis of both the water blade 20 and ultrasonic probe axes, however, inevitably, strict alignment is often not achieved once when the water blade 20 is placed. The direction of movement along the Y-axis toward the plane near the XZ-axis is the depth direction of the water blade 20, as shown in fig. 1. The ultrasonic probe located directly under the water jet 20 is provided with an ultrasonic transducer or an array of transducers on the slender shaft body thereof, and is used for detecting and forming an image of the sagittal plane 1 (i.e. the plane in which the Y axis and the Z axis are located in fig. 1) upwards, but due to operation errors, the actual pose of the water jet 20 arranged may deflect relative to the sagittal plane 1, and a certain deflection angle exists between the plane 2 in which the water jet 20 is actually located and the sagittal plane 1. In this case, the surface 2 on which the water jet blade 20 is actually located is referred to as a deflection surface 2. It will be appreciated that in the case where the water jet 20 is not deflected, the deflection angle is defined as 0. Wherein, the intersection line of the deflection surface 2 and the sagittal surface 1 is parallel to the Z axis and perpendicular to the depth direction of the water jet 20. Since the ultrasound transducer or transducer array arranged on the elongate shaft body has a certain width in the direction perpendicular to the Y-axis, this makes it possible to observe the water jet 20 and/or water jet flow images through the sagittal plane 1 defined by the ultrasound probe even if there is a deflection of the arranged water jet 20, but the observed water jet 20 and/or water jet flow images on the sagittal plane 1 defined by the ultrasound probe are not images of an ideal state due to the deflection.
As shown in fig. 1, the dashed line segment BD shows the actual pose of the water blade 20 on the deflection plane 2, and the solid line segment AC shows the ultrasound image that the water blade 20 now presents on the sagittal plane 1. In case the water jet 20 is deflected, i.e. the water jet 20 is actually in a position inclined through the sagittal plane 1, the obtained ultrasound image is information of the area indicated by the solid line segment AC, which corresponds in theory to the projection of the standard image of the water jet 20 on the sagittal plane 1, the length of the solid line segment AC being smaller than the length of the dashed line segment BD. It will be appreciated that as the angle of deflection increases, the length of the real line segment AC decreases, i.e. the corresponding imaging of the water jet 20 on the sagittal plane 1 becomes shorter and shorter.
The water jet 20 is provided with a knife hole 21 (not shown in fig. 1) through which a water jet can be injected, wherein a virtual circle W is a schematic representation of the water jet injected through the knife hole 21, and a solid circle therein is a schematic representation of an ultrasonic image of the jet on the sagittal plane 1. In the environment of water jet immersion spraying, the water jet sprayed by the knife hole 21 is gradually diffused along the Z-axis XY plane due to cavitation effect, and in the case of deflection of the water knife 20, namely, the water knife 20 is actually in a position of inclining through the sagittal plane 1, the center line of the jet is located on the deflection plane 2, and the obtained ultrasonic image is information of the area of the real coil, and the information corresponds to projection of a standard image of the jet on the sagittal plane 1 in theory. The perpendicular distance from the centerline of the jet to the sagittal plane 1 is taken as the offset distance. It will be appreciated that as the deflection angle increases, the offset distance increases and the jet image on the sagittal plane 1 becomes smaller and smaller, as indicated by the dashed circle W and the solid circle therein.
As described above, we found that in the case of deflection of the water jet 20, both the observed water jet image and the observed water jet image will show deviations, in other words, because the water jet 20 deflects relative to the sagittal plane 1 defined by the ultrasonic probe 1, the water jet 20 and the jet at the knife hole 21 cannot show ideal standard images on the sagittal plane 1, when the degree of deflection exceeds the ideal threshold, it can be determined that the condition for operating the water jet 20 is not met, at this time, the operator needs to be assisted in calculating and prompting the deflection angle, and visually displaying the actual pose of the water jet 20, so that the operator can adjust the pose of the water jet 20 according to the prompt, and further perform subsequent operations, or in the automatically controlled operation scene, feedback information such as the deflection angle to the control unit to control the pose of the water jet 20 to be undeflected.
The term "yaw, yaw angle, and offset distance" as used herein refers to a lateral yaw of the water blade 20 about an axis parallel to the Z axis with respect to the sagittal plane 1, and not a pitch yaw of the water blade 20 about an axis parallel to the X axis. It will be appreciated by those skilled in the art that for yaw of pitch up and down, the yaw information may be obtained directly in the sagittal plane 1 image, i.e., the yaw angle and/or yaw distance of pitch up and down may be obtained directly by comparing it with a standard image, which is readily available to those skilled in the art, and the present application is not further described herein.
For convenience of description, the position of the cutter head of the water cutter 20 is taken as a deflection point, namely, the water cutter 20 transversely deflects around the cutter head. In practice, the deflection point may be the orifice 21 or another position, and only the deflection around the cutter head is described here as an example. In the same way, the water column of water spray is taken as jet flow sprayed by the cutter hole 21, the practice is not particularly limited.
Deflection degree identification
Next, a method for recognizing the degree of deflection of the water jet blade 20 will be specifically described with reference to fig. 2 to 6. Fig. 2 is a schematic diagram of a preset water jet image template set 3, fig. 3 is a schematic diagram of a preset jet image template set 4, fig. 4 is a schematic diagram of the positions of two cross sections and a sagittal plane 1, fig. 5 is an imaging schematic diagram of a water jet 20 in the two cross sections, and fig. 6 is a schematic diagram of the positions of one cross section and the sagittal plane 1.
Firstly, the step of presetting the S1 image template is explained.
The preset water jet images 10 corresponding to the water jet 20 with a plurality of deflection angles on the sagittal plane 1 are obtained in advance, and a preset water jet image template set 3 is formed. As an example, through pre-experiments before operation, hardware parameters such as the water pressure of the water jet column of the water jet knife 20 and the position of the knife hole 21 are fixed, and a plurality of deflection angles of the deflection surface 2 are preset, for example, 5 groups of angles of {0 degrees, ±5 degrees, ±10 degrees, ±15 degrees, ±20 degrees } are set, and fig. 2 shows ultrasonic images acquired under the same ultrasonic depth and other acquisition parameters under different deflection angles. As can be seen from the figure, when the deflection angle is larger, the imaging of the blade body of the water jet 20, that is, the preset water jet image 10 is shorter, and the acquired images are indexed and stored between the deflection angle and the corresponding ultrasonic image to be used as the preset water jet image template set 3.
In addition, the ultrasonic images, namely the preset jet images 11, corresponding to the jet flow on the sagittal plane 1 under a plurality of offset distances are acquired in advance, so that a preset jet flow image template set 4 is formed. As an example, through pre-experiments before operation, the water pressure of the water jet column of the water jet knife 20, the position of the knife hole 21, and other hardware parameters are fixed, and a plurality of offset distances of the deflection surface 2 are preset, in this embodiment, the offset distances of {0mm,2mm,4mm,8mm,10mm }5 groups are set, and as shown in fig. 3, water jet images acquired under the same ultrasonic depth and other acquisition parameters at different offset distances, namely, preset jet images 11 are shown. As can be seen from the figure, when the offset distance is larger, the imaging of the water jet, that is, the area of the preset jet image 11 is smaller, and the images acquired by the group are indexed and stored according to the offset distance and the corresponding ultrasonic image, so as to be used as the preset jet image template set 4.
And then, analyzing by an image segmentation algorithm to obtain characteristic parameters of the image templates in the preset water jet image template set 3 and the preset jet image template set 4, such as height, width, length-width ratio, perimeter, gray value range and the like, and taking the parameters as template matching inputs for executing a template matching step. For the subsequent step, the deflection angle is conveniently and efficiently calculated and determined by taking the characteristic parameters as a part of index record information, one or more of the height, width, length-width ratio, perimeter and gray value range of the jet flow image or the water jet blade image and the jet flow contour or the water jet blade 20 contour can be selected for establishing and storing the index, and the index is directly calculated according to the parameters stored in the index when the matching is performed. In addition, a preset water jet image template set 3 and a preset water jet image template set 4 based on different ultrasonic acquisition parameters can be set for different ultrasonic acquisition parameters of the ultrasonic probe, such as ultrasonic depth, contrast and the like. And further, by changing the water pressure of the water jet columns of the water knife 20, image template sets under different water pressures can be generated based on the similar way, and the template sets of the water pressure of the different water jet columns are particularly suitable for the situation of judging the subsequent deflection degree by utilizing the preset jet image template set 4. Only one set of preset water jet image template sets 3 and one set of preset jet image template sets 4 are described here as examples, however it should be understood that the following steps are equally applicable to the case of multiple sets of preset water jet image template sets 3 or multiple sets of preset jet image template sets 4.
Next, the step of S2 image matching will be described.
As described above, the image template set of the water jet 20 with the same ultrasonic depth and other parameters is preset, the feature parameters of the template are determined by the image segmentation algorithm, and the corresponding deflection angle/offset distance, image and feature parameters are indexed and stored. As an example, the characteristic parameters of the image templates in the image template set at least comprise the position of the cutter hole 21 and the length of the water cutter 20.
After the water jet 20 is disposed in the working area 9, it is necessary to make a judgment on the degree of deflection of the water jet 20 based on the acquired ultrasonic image, and to determine the direction and magnitude of adjustment of the water jet 20 based on the judgment result. Specifically, an actual water jet image corresponding to the water jet 20 on the sagittal plane 1 is obtained, the actual water jet image is matched with an image template in the preset water jet image template set 3, and/or an actual jet image corresponding to the jet on the sagittal plane 1 is obtained, and the actual jet image is matched with an image template in the preset jet image template set 4.
When the matching result satisfies the preset first condition, it is determined that the deflection degree of the water jet 20 is acceptable without adjusting the pose of the water jet 20. At this point, the deflection of the water jet 20 is considered to be slightly unaffected by the surgical procedure, and the system may prompt the operator to make no adjustments, or only fine adjustments in the angle of deflection and/or the distance of deflection. When the matching result does not meet the preset first condition, it is determined that the deflection degree of the water jet 20 is not acceptable and the pose of the water jet 20 needs to be adjusted.
Specifically, after the actual water jet image of the water jet 20 on the sagittal plane 1 is obtained, the length AC of the water jet 20 and the position of the knife hole 21 corresponding to the jet W of the water jet water column in the image are obtained according to an image segmentation algorithm, and the index is searched for by the length of the water jet 20 and the position information of the knife hole 21 in the obtained actual water jet image, so that the image templates in the image template set and the corresponding deflection angle/deflection distance are determined, and the image templates in the determined image template set are displayed in a superimposed manner on the actual sagittal plane image, thereby facilitating the operator to intuitively see the deflection degree of the water jet 20. Alternatively, the deflection angle/offset distance information may be displayed simultaneously on the display device or transmitted to the control unit for posture adjustment of the water jet blade 20. In this way, after the water jet 20 is arranged at the predetermined position, based on the sagittal ultrasonic image acquired in real time, the deflection degree information can be quickly determined, and the image reflecting the deflection degree of the water jet 20 can be quickly called from the template set stored in the storage device, and the images are displayed in a superimposed manner on the actual sagittal 1 image in different effects (such as semitransparent or different colors, etc.), so that guidance can be provided for an operator at the first time, and the efficiency and accuracy of arranging the water jet 20 by the operator can be greatly improved.
Further, the rough deflection angle of the water jet 20 is obtained based on the matching result of the actual water jet image and the preset water jet image template set 3, and/or the rough deflection distance of the jet is obtained based on the matching result of the actual jet image and the preset jet image template set 4, wherein the first condition is that the rough deflection angle is smaller than a preset first deflection angle threshold value, and/or the rough deflection distance is smaller than a preset first deflection distance threshold value.
In particular, the angle of deflection of the water jet 20 and the distance of deflection of the jet are different characterizations of essentially the same phenomenon. That is, in the event of deflection of the water jet 20 relative to the sagittal plane 1, this necessarily results in a deflection of the jet relative to the sagittal plane 1. When the deflection surface 2 is at a certain deflection angle, the jet will also be at a corresponding offset distance. That is, the deflection angle of the water jet 20 and the offset distance of the jet are two phenomena caused by the same fact, and there is a mapping relationship between the two phenomena. However, the deflection angle of the water jet 20 and the offset distance of the jet do not necessarily match the mapping relationship as expected due to the influence of image quality, noise, and the like. For example, the deflection angle indicated by the preset water jet image template matching the actual water jet image is 5 °, while the offset distance indicated by the preset jet image template matching the actual jet image is not the offset distance corresponding to the deflection angle of 5 °.
This is because at least one of the deflection angle of the water jet 20 and the offset distance of the jet has an error due to the influence of image quality, noise, and the like. Thus, both dimensions, from the angle of deflection of the water jet 20 or the offset distance of the jet, are possible, but for a more complex imaging environment, it is more preferable to determine the degree of deflection of the water jet based on both the angle of deflection of the water jet 20 and the offset distance of the jet, so that false positives can be better avoided. In particular, after a conclusion that the deflection degree is negligible and the pose of the water jet 20 does not need to be adjusted is obtained according to one dimension judgment, the deflection degree is judged by introducing the other dimension to ensure the reliability of the conclusion. For example, when the rough deflection angle of the water jet 20 is obtained based on the matching result of the actual water jet image and the preset water jet image template set 3, and the rough deflection angle is determined to be smaller than the preset first deflection angle threshold, the system can prompt that the deflection degree of the water jet 20 is acceptable, or the system further introduces the judgment of the offset distance, namely, based on the matching result of the actual jet image and the preset jet image template set 4, the rough offset distance of the jet is obtained, and whether the rough offset distance is smaller than the preset first offset distance threshold is judged, when the rough offset distance is smaller than the preset first offset distance threshold, the system directly gives a conclusion that adjustment is not needed, and when the rough offset distance is larger than the preset first offset distance threshold, the system prompts fine adjustment. It should be understood that the above steps may be reversed, i.e., the determination of the coarse offset distance is performed first, and then the determination of the coarse offset angle is performed. And, as previously described, it is preferable that the degree of deflection of the water jet blade 20 is considered to be small without adjusting the pose of the water jet blade 20 only in the case where the rough deflection angle is smaller than the preset first deflection angle threshold value and the rough deflection distance is smaller than the preset first deflection distance threshold value.
Further, when the matching result does not meet the preset first condition, the step of calculating the deflection angle is entered into S3. In the deflection angle calculation step, it is determined whether the matching result satisfies a preset second condition. The second condition is that the rough deflection angle is smaller than a preset second deflection angle threshold value and/or the rough offset distance is smaller than a preset second offset distance threshold value.
The second deflection angle threshold is larger than the first deflection angle threshold, and the second offset distance threshold is larger than the first offset distance threshold.
In addition, the second condition may be that the degree of deflection of the water jet blade obtained by weighting the rough deflection angle and the rough deflection distance is smaller than a preset deflection degree threshold value.
When the matching result meets the preset second condition, that is, the water jet 20 needs to adjust the pose, but the deflection degree is still relatively small, the deflection angle of the water jet can be calculated directly according to the rough deflection angle and/or the rough deflection distance. This is because the water jet 20 passes obliquely through the sagittal plane 1 with the water jet 20 deflected relative to the sagittal plane 1. The greater the angle of deflection, the smaller the projection of the water jet 20 onto the sagittal plane 1, and the further away the jet is from the sagittal plane 1, and therefore the lower the accuracy of the rough angle of deflection determined based on the image of the water jet 20 onto the sagittal plane 1 and the rough distance of deflection determined based on the image of the jet onto the sagittal plane 1. Conversely, the smaller the deflection angle, the higher the accuracy of the coarse deflection angle determined based on the image of the water jet 20 on the sagittal plane 1 and the coarse offset distance determined based on the image of the jet on the sagittal plane 1.
Thus, when it is determined that the pose of the water jet 20 needs to be adjusted, the second condition is a condition for determining whether or not the deflection angle of the water jet 20 is small, so that the rough deflection angle of the water jet 20 and the rough offset distance of the jet flow are sufficient to accurately calculate the deflection angle of the water jet 20.
When the matching result does not satisfy the preset second condition, the deflection angle of the water jet 20 is relatively large, and the rough deflection angle of the water jet 20 and the rough offset distance of the jet are insufficient to accurately determine the deflection angle of the water jet 20. In this case, it is necessary to introduce a cross-sectional image to perform more accurate angle calculation and to determine the deflection angle of the water jet 20 via a cross-section of at least one perpendicular to the sagittal plane 1. The cross section of the ultrasonic probe is perpendicular to the depth direction of the ultrasonic probe, namely the Y-axis direction.
The acquisition of the cross-sectional image requires the movement of the ultrasound probe, which may put a burden on the patient. With the present invention, the calculation of the deflection angle of the water jet 20 is performed based on the sagittal plane image, in the case that the deflection angle of the water jet 20 is determined accurately by the rough deflection angle of the water jet 20 and the rough offset distance of the jet. Only when the deflection angle of the water jet 20 is large, the deflection angle of the water jet 20 is calculated based on the cross-sectional image, and thus the burden on the patient can be reduced to the maximum extent.
In addition, the deflection angle of the water jet 20 calculated through the above steps is usually required to be presented to the operator by means of visual display or the like, so as to help the operator quickly complete the pose adjustment of the water jet 20 according to the presented deflection angle (corresponding to the angle to be adjusted). The angle of deflection of the water jet 20 calculated from the cross-sectional image is generally relatively precise (the details will be described later), while the accuracy of the angle of deflection of the water jet 20 calculated from the sagittal image depends on the magnitude of the angle of deflection.
Specifically, the smaller the angle of deflection of the water jet 20, the higher the accuracy of the rough angle of deflection determined based on the image of the water jet 20 on the sagittal plane 1 and the rough distance of offset determined based on the image of the jet on the sagittal plane 1. As described above, the deflection angle of the water jet 20 and the offset distance of the jet are two phenomena due to the same fact, and there is a mapping relationship between the two phenomena. The smaller the deflection angle of the water jet 20, the higher the accuracy of the rough deflection angle and the rough offset distance, the higher the probability that both conform to the mapping relationship, and the higher the reliability of the deflection angle of the water jet 20 calculated through the two dimensions.
On the contrary, when the rough deflection angle of the water jet 20 and the rough deflection distance of the jet flow match the mapping relationship, the reliability of the determined deflection angle is higher. During pose adjustment of the water jet 20, it is desirable to move the ultrasonic probe and the water jet 20 as little as possible to reduce the burden on the patient. Therefore, when the rough deflection angle of the water jet 20 and the rough deflection distance of the jet flow accord with the mapping relationship, the determined deflection angle is more accurate, which is beneficial to prompting the operator to adjust the water jet 20 to a desired state through one-time operation. On the other hand, if the rough deflection angle of the water jet 20 and the rough deflection distance of the jet do not conform to the mapping relationship, at least one of the two may have a larger error, the operator may be prompted to heuristically adjust the water jet 20, and then determine the deflection degree and determine the deflection angle again until the rough deflection angle of the water jet 20 and the rough deflection distance of the jet conform to the mapping relationship, and the water jet 20 is adjusted again. Thus, the burden on the patient can be reduced as much as possible while ensuring the accuracy of adjustment.
The method steps for determining the deflection angle by introducing cross-sectional image information are described further below.
In the deflection angle calculating step, under the condition that the matching result does not meet the second condition, if the matching result meets a preset third condition, acquiring two cross section ultrasonic images which are arranged at intervals, determining a corresponding linear equation of the water jet 20 in a physical space coordinate system based on the positions of imaging points of the water jet 20 in the two cross section images, and if the matching result does not meet the third condition, acquiring only an ultrasonic image of one cross section, and determining a corresponding linear equation of the water jet 20 in the physical space coordinate system based on the positions of imaging points of the water jet 20 in the cross section images and the positions of imaging points of the jet holes 21 in the sagittal plane images.
The third condition is that the rough deflection angle is larger than a preset third deflection angle threshold value and/or the rough deflection distance is larger than a preset third deflection distance threshold value. The third deflection angle threshold is greater than the second deflection angle threshold, and the third offset distance threshold is greater than the second offset distance threshold.
Specifically, in the step of calculating the deflection angle in S3, when the matching result does not satisfy the second condition and satisfies the third condition, it is explained that the deflection angle of the water jet blade 20 is relatively large, and it is necessary to introduce two cross-sectional images for judgment.
Referring to fig. 4 and 5, ultrasonic images of a first cross section 51 and a second cross section 52 which are arranged at intervals are acquired, and the ultrasonic images of the water jet 20 at the cross sections corresponding to the first cross section 51 and the second cross section 52 are respectively presented as a first imaging point B1 and a second imaging point B2. Because the cross section is approximately perpendicular to the blade body of the water blade 20, the image of the water blade 20 on the ultrasonic cross section approximately shows the cross section information of the shaft body of the water blade 20, more often is similar to a circle, and for convenience of description, the cross section is expressed as an imaging point, and it should be understood that the imaging point can be the fitting circle center of the outline of the similar circular water blade 20. Based on the positions of the first imaging point B1 and the second imaging point B2, a corresponding linear equation of the water jet 20 in the physical space coordinate system is determined.
Next, the actual pose of the water jet 20 in the physical space coordinate system can be determined along the straight line direction determined by the straight line equation and the jet position of the water jet 20. Furthermore, the corresponding image templates in the image template set can be selected and displayed in a superimposed manner in the sagittal image, so that an operator can intuitively see the deflection degree of the water jet 20 and acquire relevant information for subsequent adjustment.
As an example, a cross-sectional image is acquired by selecting a position corresponding to the knife hole 21 in the sagittal plane 1 image, and the image of the knife hole 21 in the sagittal plane 1 image is taken as the position of the first cross-section 51. As described in the scheme of the present inventors' prior application for the follow-up display of sagittal plane 1 images and transverse images, after the ultrasound probe is placed in place, the operator has achieved the follow-up display of sagittal plane 1 images and transverse images, and thus the determination of the position is readily available. After determining the acquisition location of the first cross-sectional image, the ultrasound probe is driven to translate or acquire a cross-sectional image of the cross-sectional location by fitting the images stored in the storage device adjacent to the cross-sectional location.
Further, based on the position of the first cross section 51, a suitable position of the second cross section 52 is selected, and a second cross section image of the position of the second cross section 52 is acquired, and a method for acquiring the second cross section image is similar to that for acquiring the first cross section image, which is not described herein.
The calculation accuracy and the safety factor may be set in advance, if the calculation accuracy is 5mm and the safety factor is 6 times, the safety accuracy m=5 mm×6=30 mm, and the distance interval between the first cross section 51 and the second cross section 52 in the Y-axis direction is set to 30mm. The setting of the safety factor is not too small or too large, on the one hand, considering the errors of the image recognition and the ultrasound image, the too close spacing of the first cross section 51 and the second cross section 52 will affect the accuracy of the deflection angle judgment, and on the other hand, considering that in some situations the ultrasound probe needs to be driven to move to obtain the cross section image of the position of the second cross section 52, the movement amplitude is not desired to be too large to avoid accidents or damages, and the safety factor is not desired to be too large, and therefore, the distance between the first cross section 51 and the second cross section 52 is determined based on the errors of the image recognition and the ultrasound image, and the movement amplitude threshold value desired to be controlled.
In some embodiments, the first and second cross sections 51, 52 are positioned at Y-axis coordinates Y1, Y2, respectively. According to the maximum deflection angle, for example, 20 degrees, set correspondingly for the image templates in the preset water jet image template set 3, y2=y1+m×tan20. The calibration relation between the ultrasonic image coordinate system and the physical space coordinate system is established in advance, the coordinates of the y1 and the y2 are converted into the coordinates of the physical space coordinate system from the ultrasonic image coordinate system, and the ultrasonic probe is respectively moved to the positions corresponding to the y1 and the y2 so as to obtain the cross-sectional images of the positions of the first cross section 51 and the second cross section 52 shown in fig. 5.
As a preferable mode, the first pixel coordinate and the second pixel coordinate of the first imaging point B1 and the second imaging point B2 in the ultrasonic image coordinate system are respectively obtained through an image segmentation algorithm, then the first pixel coordinate and the second pixel coordinate are converted into the first space coordinate and the second space coordinate in the physical space coordinate system according to the calibration relation, and the linear equation of the water jet cutter 20 can be determined according to the first space coordinate and the second space coordinate.
Referring to fig. 6, in the case where the matching result does not satisfy the aforementioned second condition and third condition, it is illustrated that although the reliability of the determined deflection angle of the water jet blade 20 is low, the deflection degree thereof is still relatively small, and in this case, in order to improve the calculation efficiency and reduce the probe movement, only one cross section may be introduced for determining the deflection angle.
The specific method is similar to the principle that the first cross section 51 and the second cross section 52 are obtained and the coordinates of the first imaging point B1 and the second imaging point B2 are determined, and then the straight line equation of the water jet 20 is determined, unlike the principle that the first imaging point B1 is replaced by the imaging point corresponding to the knife hole 21 in the sagittal plane image, that is, the cross section image corresponding to the position of the first cross section 51 is not required to be obtained, and the cross section image corresponding to the position of the second cross section 52 is only required to be obtained to determine the second imaging point B2, so that the calculation process is more efficient, the moving times of the ultrasonic probe can be reduced, that is, the ultrasonic probe is only required to be moved to the position corresponding to the second cross section 52, and the ultrasonic probe is not required to be moved to the position corresponding to the first cross section 51, thereby not only improving the efficiency, but also further reducing the damage to the cavity tissue.
Further, the pixel coordinates of the second imaging point B2 and the imaging point of the knife hole 21 in the sagittal plane image are obtained through an image segmentation algorithm, and then the pixel coordinates are converted into physical space coordinates according to the calibration relation between the ultrasonic image coordinate system (which may be a cross-sectional image coordinate system and a sagittal plane image coordinate system) and the physical space coordinate system, so that the linear equation of the water knife 20 in the physical space coordinate system can be determined. Based on the determined straight line direction and the position of the cutter hole of the water cutter 20, the actual pose of the water cutter 20 in the physical space coordinate system can be determined. Furthermore, the corresponding image templates in the image template set can be selected and displayed in a superimposed manner on the sagittal image, so that the operator can intuitively see the deflection degree of the water jet 20 and acquire relevant information for subsequent adjustment.
In summary, according to the method for identifying the deflection degree of the water jet blade of the present application, firstly, the ultrasonic image of the water jet blade 20 and/or the jet flow on the sagittal plane 1 is matched with the pre-established image template, so as to primarily determine the deflection angle of the deflection plane 2 relative to the sagittal plane 1 and/or the deflection distance of the jet flow relative to the sagittal plane 1. If the first condition is met, the degree of deflection is considered acceptable. Otherwise, judging whether the second condition is satisfied.
If the first condition is not satisfied, the water jet 20 is considered to have a relatively small deflection degree although the pose needs to be adjusted if the second condition is satisfied, the deflection angle of the water jet can be calculated directly from the rough deflection angle and/or the rough offset distance, and if the second condition is not satisfied, it is determined whether the third condition is satisfied.
Under the condition that the second condition is not met, if the third condition is met, the deflection degree is serious, information in cross-sectional images of two cross-sectional positions is required to be introduced to increase the accuracy of deflection angle calculation, and if the third condition is not met, only one cross-section is required to be introduced to judge, so that the moving times and the distance of an ultrasonic probe are reduced, and the damage to the cavity tissue is reduced.
The deflection degree identification method of the water knife can not only efficiently and accurately determine the deflection degree information of the water knife, but also introduce different judgment strategies according to different deflection degrees, thereby further considering efficiency and accuracy. Through the information superposition utilization of the ultrasonic sagittal image and the cross-section image, the determination of the deflection angle of the water knife 20 has higher reliability, and simultaneously, the water knife 20 and the deflection information thereof are visually displayed, so that an operator can efficiently and accurately adjust the pose of the water knife 20.
Meanwhile, when the deflection degree of the water jet 20 is acceptable, it is not necessary to introduce an image of the cross-sectional position, i.e., it is not necessary to move the ultrasonic probe. When the deflection degree of the water jet 20 is not acceptable, if the deflection angle is relatively not large, only one image of the cross section position is required to be introduced, namely the ultrasonic probe moves once along the Y axis, and only when the deflection angle is judged to be large, only two images of the cross section position are required to be introduced, namely the ultrasonic probe moves twice along the Y axis. Therefore, the method also takes the accuracy of deflection angle judgment into consideration, reduces the influence on the tissues in the patient, and avoids possible damage.
It should be noted that, only the actual water jet image may be selected to be matched with the preset water jet image template set 3 to obtain a rough deflection angle, or the actual jet image may be selected to be matched with the preset jet image template to obtain a rough offset distance, and then the first condition, the second condition and the third condition may be determined according to the rough deflection angle or the rough offset distance. As described above, the deflection angle and the offset distance are different representations of the same phenomenon, and have a certain corresponding relationship, and meanwhile, the rough deflection angle and the rough offset distance are obtained and judged, which is only one preferred embodiment of the application.
Further, it should be understood that the storage devices referred to in the above embodiments may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an erasable programmable ROM (erasable PROM), an electrically erasable programmable EPROM (EEPROM), or a flash memory. The volatile memory may be random access memory (random access memory, RAM) which acts as external cache memory. By way of example, and not limitation, many forms of random access memory (random access memory, RAM) are available, such as static random access memory (STATIC RAM, SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (double DATA RATE SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCHLINK DRAM, SLDRAM), and direct memory bus random access memory (direct rambus RAM, DR RAM).
And the embodiments described above may be implemented in whole or in part by software, hardware (e.g., circuitry), firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with embodiments of the present invention are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, from one website site, computer, motor drive, or data center to another website site, computer, motor drive, or data center by infrared, wireless, microwave, or the like. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a motor drive, data center, or the like, that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.
It can be appreciated, therefore, that the present application also provides a water jet deflection degree recognition system. The system comprises an image acquisition module, an image template matching module and a water knife deflection degree calculation module, wherein the image acquisition module is used for acquiring ultrasonic images of the water knife 20 and jet flow thereof, the image template matching module is used for matching the ultrasonic images of the water knife 20 and jet flow thereof acquired by the image acquisition module with image templates in a preset water knife image template set 3 and a preset jet flow image template set 4, and the water knife deflection degree calculation module is used for determining the deflection degree of the water knife 20 according to the water knife deflection degree identification method. And the deflection degree recognition system of the water jet 20 provided by the application is suitable for executing the method steps, and all the systems for realizing the method steps are within the protection scope of the application.
It should be understood that the above embodiments are only for explaining the present application, the protection scope of the present application is not limited thereto, and any person skilled in the art should be able to modify, replace and combine the technical solution according to the present application and the inventive concept within the scope of the present application.