Disclosure of Invention
The embodiment of the application provides a bleeding detection method, a bleeding detection system and image processing equipment, which are used for at least solving the problem of low real-time performance and accuracy of bleeding condition detection in the related technology.
In a first aspect, an embodiment of the present application provides a method for detecting bleeding, the method including:
Acquiring a first image to be detected of a target tissue acquired by an endoscope;
determining the bleeding amount of the target tissue according to the image gradient of the first image to be detected;
Acquiring a plurality of frames of second images to be detected of the target tissue acquired by the endoscope before and/or after the first images to be detected under the condition that the bleeding amount of the target tissue is the target bleeding amount;
and determining the bleeding severity of the target tissue according to the change information of the color contrast between at least two frames of first target images, wherein the first target images are images in the first image to be detected and the plurality of frames of second images to be detected.
In some of these embodiments, the method further comprises:
And when the image gradient is smaller than a preset gradient change threshold value, judging that the bleeding condition of the target tissue is the target bleeding amount condition.
In some embodiments, the determining the bleeding severity of the target tissue according to the change information of the color contrast between the at least two frames of the first target image includes:
Performing color space conversion processing on the first target image of each frame to obtain an LAB image in an LAB color space and characteristic color dimension information of the LAB image;
And determining the change information of the color contrast between the LAB images of each frame according to the characteristic color dimension information, and determining the bleeding severity according to the change information of the color contrast between the LAB images.
In some embodiments, the determining the bleeding severity of the target tissue according to the change information of the color contrast between the at least two frames of the first target image includes:
Generating first color change trend information corresponding to a second target image in the first image to be detected and the plurality of frames of second images to be detected according to the color contrast of the second target image;
determining bleeding amount information of the target tissue according to the first color change trend information;
And determining the bleeding severity of the target tissue according to the bleeding amount information and the change information of the color contrast between the at least two frames of first target images.
In some embodiments, the determining the bleeding amount information of the target tissue according to the first color change trend information includes:
Determining a first color contrast to be detected corresponding to the number of target pixel points in the second target image according to the first color change trend information;
and under the condition that the first to-be-detected color contrast is detected to be in the preset color area range, calculating first concentration degree information among all the first to-be-detected color contrast, and determining bleeding amount information of the target tissue according to the first concentration degree information.
In some of these embodiments, the second target image is a most recent frame image of the first to-be-detected image and the number of frames of second to-be-detected images.
In some embodiments, the determining the bleeding severity of the target tissue according to the change information of the color contrast between the at least two frames of the first target image includes:
generating second color change trend information corresponding to the first target image according to the color contrast of the first target image of each frame;
Determining the change information of the color contrast according to the change information among all the second color change trend information, and determining the bleeding speed information of the target tissue according to the change information of the color contrast;
determining a bleeding severity of the target tissue from the bleeding speed information.
In some embodiments, the determining the change information of the color contrast according to the change information among all the second color change trend information includes:
Determining a second color contrast to be detected corresponding to the number of target pixel points in each frame of the first target image according to the second color change trend information;
under the condition that the second color contrast to be detected is detected to be in a preset color area range, calculating second concentration degree information among all the second color contrast to be detected in each frame of the first target image;
And determining the change information of the color contrast according to the change information between the second concentration degree information corresponding to the first target image in each frame.
In some of these embodiments, the image acquisition mode of the endoscope includes a white light mode and a non-white light mode, and after the determining the bleeding severity of the target tissue, the method further includes:
under the condition that the bleeding severity is the target severity, switching the image acquisition mode from the white light mode to the non-white light mode, and acquiring a non-white light image of the endoscope in the non-white light mode, wherein the light irradiated by the endoscope to the target tissue in the white light mode is white light, and the light irradiated to the target tissue in the non-white light mode is non-white light;
And generating a bleeding position detection result aiming at the target tissue according to the non-white light image.
In some of these embodiments, after the determining the bleeding severity of the target tissue, the method further comprises:
Generating bleeding position marker information for the target tissue based on the bleeding position detection result;
And generating an image to be displayed according to the bleeding position mark information and the white light image.
In a second aspect, an embodiment of the present application provides a bleeding detection system including a first acquisition portion, a first determination portion, a second acquisition portion, and a second determination portion;
The first acquisition part is used for acquiring a first image to be detected of target tissue acquired by the endoscope;
The first determining part is used for determining the bleeding amount condition of the target tissue according to the image gradient of the first image to be detected;
The second acquisition part is used for acquiring a plurality of frames of second images to be detected of the target tissue acquired by the endoscope before and/or after the first images to be detected under the condition that the bleeding amount of the target tissue is the target bleeding amount;
the second determining part is used for determining the bleeding severity degree of the target tissue according to the change information of the color contrast between at least two frames of first target images, wherein the first target images are images in the first image to be detected and the plurality of frames of second images to be detected.
In a third aspect, an embodiment of the present application provides an image processing apparatus, the memory storing a computer program, the processor being configured to run the computer program to perform the bleeding detection method according to the first aspect.
In a fourth aspect, an embodiment of the present application provides an endoscope system including a display, an endoscope, a light source device, and an image processing device as described in the third aspect above.
Compared with the related art, the bleeding detection method, the bleeding detection system and the image processing device provided by the embodiment of the application have the advantages that the first to-be-detected image of the target tissue collected by the endoscope is obtained, the bleeding condition of the target tissue is determined according to the image gradient of the first to-be-detected image, a plurality of frames of second to-be-detected images of the target tissue collected by the endoscope before and/or after the first to-be-detected image is obtained under the condition that the bleeding condition of the target tissue is the target bleeding condition, the bleeding severity of the target tissue is determined according to the change information of the color contrast between at least two frames of first to-be-detected images, and the first to-be-detected image is the image in the first to-be-detected image and the frames of second to-be-detected images, so that the bleeding condition of the bleeding point can be detected in time when the bleeding point appears, the bleeding condition of the bleeding point can be detected in real time, the bleeding condition can not be detected in time due to the fact that the image collection of the endoscope is not in time is processed, or the bleeding detection accuracy is low is avoided, the problem that the bleeding condition is detected in time is solved, and the bleeding condition is detected in time is low is solved, and the timely bleeding condition is detected, and the high efficiency is achieved.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the other features, objects, and advantages of the application.
Detailed Description
The present application will be described and illustrated with reference to the accompanying drawings and examples in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application. All other embodiments, which can be made by a person of ordinary skill in the art based on the embodiments provided by the present application without making any inventive effort, are intended to fall within the scope of the present application. Moreover, it should be appreciated that while such a development effort might be complex and lengthy, it would nevertheless be a routine undertaking of design, fabrication, or manufacture for those of ordinary skill having the benefit of this disclosure, and thus should not be construed as having the benefit of this disclosure.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly and implicitly understood by those of ordinary skill in the art that the described embodiments of the application can be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. The terms "a," "an," "the," and similar referents in the context of the application are not to be construed as limiting the quantity, but rather as singular or plural. The terms "comprises," "comprising," "includes," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to only those steps or elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "connected," "coupled," and the like in connection with the present application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as used herein means greater than or equal to two. "and/or" describes the association relationship of the association object, and indicates that three relationships may exist, for example, "a and/or B" may indicate that a exists alone, a and B exist simultaneously, and B exists alone. The terms "first," "second," "third," and the like, as used herein, are merely distinguishing between similar objects and not representing a particular ordering of objects.
The endoscope refers to an optical device which enters the human body through a natural duct of the human body or through a small incision made by operation to observe a relevant area. Endoscopes of different lengths and diameters can be used to detect different regions of the human body, and according to the difference of the detection regions, endoscopes can be classified into a plurality of types such as laparoscopes, neuroendoscopes, arthroscopes, esophagoscopes and the like. In the operation, a great amount of bleeding can be caused by operation injury to submucosal vessels of target tissues of the object to be examined, and the flowing blood covers more and more areas of the image collected by the endoscope, so that the condition of the target tissues in the operation is influenced by operators in observation from the images.
For detection of bleeding in an intraoperative target tissue, in an alternative embodiment, the bleeding condition and the location of the bleeding point may be determined by an operator viewing the condition in the endoscopic image. However, for operators, the blood vessels of the submucosa of the target tissue are abundant and difficult to find, especially when the bleeding speed of the bleeding point is high, the blood may not cover a large area of the endoscope image at the current moment, and the bleeding point of the target tissue is not judged to be needed to be subjected to the hemostasis operation, and often the bleeding point of the target tissue is not detected until the blood begins to fill a large area of the endoscope image, so that the bleeding condition of the target tissue is difficult to accurately detect in time, and the operation is affected.
In order to solve the problem of low real-time performance and accuracy of bleeding condition detection, the bleeding detection method provided by the application carries out image recognition on an endoscopic image obtained by shooting the endoscopic insertion part penetrating into the target tissue through an artificial intelligence computer vision technology so as to determine the bleeding severity of the target tissue based on the image color information correspondingly represented by the endoscopic image. Among them, the method is applicable to various application environments of medical endoscope systems, for example, a medical endoscope system related to a gastroscope or an endoscope system related to a enteroscope, and the like. And, the method can be applied to a host of an endoscope system, that is, an image processing apparatus. Based on this, the embodiment of the present application does not limit the application environment and execution subject of the method. Illustratively, referring to FIG. 5, the application environment includes an endoscopic system of a display 52, an endoscope 54, a light source device 56, and an image processing device 58. On the basis, the first to-be-detected image and the second to-be-detected image shot by the endoscope are executed by the image processing device 58, so that timely and accurate detection of the bleeding condition and the bleeding severity of the target tissue is realized.
The present embodiment provides a bleeding detection method, and fig. 1 is a flowchart of a bleeding detection method according to an embodiment of the present application, and as shown in fig. 1, the flowchart includes the following steps:
step S110, a first to-be-detected image of the target tissue acquired by the endoscope is acquired.
In performing an examination of the digestive tract of a subject by using an endoscope, the endoscope insertion portion can be inserted into the digestive tract to be examined. In the embodiment of the present application, the digestive tract portion to be detected by the endoscope is referred to as target tissue. For example, in an enteroscopy procedure, an enteroscopy insertion portion is inserted into the intestine of an object to be examined, at which time the target tissue includes the intestine. Also for example, during an examination of the stomach with a gastroscope, the gastroscope insertion portion is inserted into the stomach, where the target tissue comprises the stomach.
The first image to be detected refers to a single frame endoscopic image of the target tissue, which is acquired at a certain time in the process of checking the target tissue by the endoscope. After the endoscope insertion portion is inserted into the target tissue, the endoscope head end portion acquires information inside the target tissue, and the first image to be detected can be obtained.
Step S120, determining the bleeding amount of the target tissue according to the image gradient of the first image to be detected.
Wherein, the bleeding amount refers to whether the detected target tissue bleeds or whether the bleeding is serious. If the bleeding amount conditions possibly occurring in the target tissue can be divided in advance, the divided bleeding amount conditions include a first-stage bleeding amount condition, a second-stage bleeding amount condition and a third-stage bleeding amount condition. The first level of bleeding is used to indicate a moderate bleeding of the target tissue, the second level of bleeding is used to indicate a slight bleeding of the target tissue, and the third level of bleeding is used to indicate a small bleeding or no bleeding of the target tissue. The dividing threshold values of the three bleeding amount conditions can be determined according to the condition of the target tissue in practical application, and are not limited herein. It can be understood that the above-mentioned pre-divided bleeding amount conditions may be further added to indicate severe bleeding of the target tissue according to practical applications, or the range of each bleeding amount condition may be further divided more finely, or the two types of bleeding may be divided into two types, i.e. bleeding and non-bleeding, which are not described herein. And then determining the range of preset bleeding amount conditions which fall into the actual bleeding amount conditions of the target tissue at corresponding moments according to the detected image gradient of the first to-be-detected image.
Specifically, the gray value of each pixel point in the first image to be detected is counted, and the change speed of the gray value of the whole image in the first image to be detected in the frame is counted, so that the image gradient of the first image to be detected can be obtained. Then, based on the image gray value change speed of the first image to be detected, detecting whether the target tissue has bleeding such as blood vessel rupture or not at the corresponding moment when the first image to be detected is acquired, for example, when the degree of difference of image gradients in the first image to be detected is smaller, indicating that the gray values of all pixel points tend to be consistent in the frame image, or when the gray gradient value of the corresponding color channel, such as an R channel, of the first image to be detected is smaller, indicating that the corresponding moment when the first image to be detected is acquired may cause blood vessel rupture, namely, at the moment, blood trace appears in the first image to be detected, and the amount of the blood trace in the picture is positively correlated with the image gradient information, so that the corresponding bleeding amount can be determined.
It can be appreciated that in this embodiment, an image gradient of at least one frame of the first image to be detected may be detected to determine the bleeding amount under the corresponding frame. If the image gradient indication condition of the first to-be-detected image is normal, for example, the first to-be-detected image is corresponding to the detected bleeding amount condition of the first to-be-detected image, each frame of image can be continuously collected and monitored in real time through the steps, and if the image gradient indication condition of the first to-be-detected image is abnormal, for example, the first to-be-detected image is corresponding to the detected bleeding amount condition of the second to-be-detected image or the third to-be-detected image, the bleeding condition of the target tissue is further detected in the subsequent steps based on the first to-be-detected image of the frame.
Step S130, in the case that the bleeding amount of the target tissue is the target bleeding amount, a plurality of frames of second to-be-detected images of the target tissue acquired by the endoscope before and/or after the first to-be-detected images are acquired.
The above-mentioned target bleeding amount refers to a case where the target tissue is ruptured in blood vessels and the bleeding amount is large at a corresponding time, and the treatment is required by a subsequent step. Taking the above-mentioned primary bleeding amount case, secondary bleeding amount case and tertiary bleeding amount case as examples, the bleeding amount case of the target tissue has been divided in advance, since the primary bleeding amount case and the secondary bleeding amount case are used to indicate that the target tissue has bleeding points, and the target tissue is in a case of moderate bleeding or light bleeding, it may have been necessary to draw attention of an operator to pay attention to the real-time case of the bleeding points of the target tissue in operation, so the primary bleeding amount case and the secondary bleeding amount case may be preset as the above-mentioned target bleeding amount case. When the first to-be-detected image is used for detecting that the target tissue bleeding amount is in the first-stage bleeding amount condition and the second-stage bleeding amount condition, the corresponding bleeding amount condition is determined to be the target bleeding amount condition, and correspondingly, the third-stage bleeding amount condition is preset to be the non-target bleeding amount condition. Or the bleeding amount condition of other bleeding amount degree, such as the bleeding amount condition indicating heavy bleeding of the target tissue, can be preset as the target bleeding amount condition according to the actual application. Or the three-level bleeding amount situation can be divided again, the bleeding amount situation which is used for indicating that the target tissue is in the range of 5-10% of the total blood amount of the whole body in the three-level bleeding amount situation is preset and is not the target bleeding amount situation, and correspondingly, the situation which is used for indicating that the target tissue is not bleeding in the three-level bleeding amount situation can be preset as the non-target bleeding amount situation.
When the condition that the bleeding amount of the target tissue is in the target bleeding amount is detected, the bleeding amount is larger at the corresponding moment, and the specific condition of the bleeding amount needs to be further and more accurately detected. Specifically, in practical application, bleeding is a phenomenon that lasts for a period of time, so that specific situations of bleeding, such as bleeding speed and bleeding amount, can be analyzed based on blood change information in a plurality of frames of images. Based on the above, when determining that the target bleeding amount exists in the first image to be detected, the embodiment of the application can acquire the image acquired near the first image to be detected to perform information analysis on the time dimension, so as to determine the required bleeding condition data. More specifically, the multi-frame endoscopic image may be acquired within a time period before the time corresponding to the first to-be-detected image is acquired, so as to obtain a second to-be-detected image before the first to-be-detected image, or the acquired multi-frame endoscopic image may be used as the second to-be-detected image within a period of time after the time corresponding to the first to-be-detected image is acquired, or each frame of endoscopic image within a certain time period including the time when the first to-be-detected image is acquired may be used as the second to-be-detected image. It can be seen that there is a temporal correlation between the first image to be detected and the second image to be detected of each frame, so as to further follow up the fine detection flow.
And step 140, determining the bleeding severity of the target tissue according to the change information of the color contrast between at least two frames of first target images, wherein the first target images are images in the first to-be-detected image and the plurality of frames of second to-be-detected images.
Specifically, a plurality of frames of images for further detecting the bleeding severity of the target tissue may be determined from the first image to be detected and the second image to be detected, and used as the first target image. For example, at least one of the first image to be detected and the plurality of frames of the second image to be detected may be used as the first target image, or at least two frames of images may be determined as the first image to be detected from only the plurality of frames of the second image to be detected. It can be understood that, in the at least two frames of first target images, each frame of first target image may be an adjacent frame of image, or each frame of first target image may be an image with a determined interval between each frame of first target image. For example, two frames of images may be spaced between a first frame of a first target image and a second frame of the first target image. Or in the at least two frames of first target images, adjacent frame images can be arranged between part of the first target images, and images with determined intervals can be arranged between part of the first target images.
After the multi-frame first target image is determined, counting the color contrast information in each frame of first target image, further calculating information such as the change rate or the change degree value of the color contrast between the multi-frame first target image, and determining the numerical information as the change information of the color contrast between the multi-frame first target image.
It should be noted that, when a bleeding point has occurred in the target tissue during surgery, if the operator has not performed operations such as hemostasis on the tissue in time, the target tissue may continuously bleed for a period of time, which results in a change in color contrast between multiple frames of the first target image during the period of time. When the change rate or the change degree of the color contrast between the multi-frame images is higher, for example, the change rate of the color contrast between two frames reaches more than 50%, and/or the change degree value reaches more than 50%, the blood spreading speed of the bleeding point of the target tissue in the corresponding time period is higher, so that the change of the color contrast between the front frame image and the rear frame image is higher, namely, the bleeding speed of the target tissue in the time period is higher, and conversely, when the change rate or the change degree of the color contrast between the multi-frame images is lower, the blood spreading speed of the bleeding point of the target tissue in the corresponding time period is lower, namely, the bleeding speed of the target tissue in the time period is lower. Therefore, the bleeding speed among the multiple frames can be determined according to the change speed of the color contrast among the multiple frames. Specifically, when the bleeding speed is detected to be high, the wound area of the bleeding point of the target tissue is large, the bleeding of the target tissue is serious, and when the bleeding speed is detected to be low, the corresponding bleeding severity of the target tissue is low, so that the bleeding severity of the target tissue is finally determined.
Through the steps S110 to S140, whether the bleeding amount of the target tissue is detected or not is preliminarily determined through the image gradient of the first image to be detected, and the bleeding condition can be detected in time when the bleeding point occurs. Under the condition that the target bleeding amount is determined, the bleeding speed of the bleeding point is detected in real time through the change information of the color contrast between the first image to be detected and the first image in the plurality of frames of the second images to be detected, so that the situation that when the bleeding amount is not large, but the blood spreading speed is not detected in time when the bleeding amount is very high, a large amount of areas of the endoscope image are filled with blood to influence the operation is avoided, and the timeliness of the bleeding detection can be improved through detecting the bleeding speed in real time. In addition, the data processing of the bleeding detection method is relatively simple, and the number of required image frames is small, so that the processing load is low, the speed is high, and the real-time performance of bleeding detection can be effectively improved. Meanwhile, through carrying out accurate quantitative analysis on image information acquired by an endoscope, namely determining the bleeding amount condition through image gradients and quantitatively detecting the bleeding speed through the change information of color contrast between each frame of images, the phenomenon that whether bleeding is needed to be stopped or not to cause low bleeding detection accuracy through manual observation of the images and judgment based on experience is avoided, and the bleeding condition detection accuracy of target tissues is improved. Therefore, the embodiment solves the problem of low real-time performance and accuracy of bleeding condition detection through the steps, and realizes a timely and efficient bleeding point detection method.
In some embodiments, the method further comprises determining that the bleeding condition of the target tissue is the target bleeding amount condition when the image gradient is less than a preset gradient change threshold. When the gradient is detected to be reduced to the preset gradient change threshold value, the condition that the blood vessel of the target tissue is broken in the operation at the current moment is judged, so that a blood trace exists in the first image to be detected, namely the bleeding condition of the target tissue is judged to be the target bleeding amount condition.
It should be noted that, the target bleeding amount condition may be adjusted based on the preset gradient change threshold and combined with an actual application condition. For example, when it is required to detect that the target tissue has just appeared at the bleeding point through the preliminary detection step, that is, when the bleeding amount is relatively small, that is, the subsequent fine detection step is performed, the gradient change threshold may be set to the first threshold. It should be further noted that in the related art, in the operation of using an endoscope to probe into the target tissue, since the blood vessels of the submucosal layer of the target tissue are abundant and are not easy to find, the problem that the bleeding point is difficult to detect in time is generally existed, while in the embodiment, the whole image gradient of the first image to be detected is monitored in real time by combining the first threshold, so that the bleeding point of the target tissue at the current moment is easy to find in time, and corresponding measures can be taken in time. Or when it is detected in the above step that the bleeding amount of the target tissue in the first image to be detected reaches a certain amount, and then the subsequent fine detection processing is performed, the gradient change threshold may be set to a second threshold, where the second threshold is greater than the first threshold. It can be understood that in this embodiment, gradient change thresholds of different gear positions may be set according to actual situations, which will not be described herein.
Through the embodiment, the bleeding condition of the target tissue is determined through the preset gradient change threshold value, so that the accuracy of bleeding detection of the target tissue is improved, meanwhile, different bleeding amount condition dividing ranges can be determined through setting different gradient change threshold values, and the sensitivity of the bleeding detection method to the initial detection result of the target tissue is adjusted, so that the bleeding detection method is more flexible.
In some embodiments, the determining the bleeding severity of the target tissue according to the color contrast variation information between the at least two first target images includes the following steps:
Firstly, performing color space conversion processing on the first target image of each frame to obtain an LAB image in an LAB color space and characteristic color dimension information of the LAB image. It should be noted that, through the steps described above, the first image to be detected or the second image to be detected acquired by the endoscope is usually an original image in RGB format. Therefore, in the embodiment, in the process of performing the fine detection of the bleeding condition of the target tissue, the first target image originally in the RGB color space can be subjected to the color gamut conversion processing so as to convert the first target image from the RGB color space to the LAB color space with a relatively wider range in the red color channel, so that the detection result can be more obvious. Next, for the LAB image in the LAB color space, color contrast in the red channel is detected, for example, an a value of all pixels in each frame of LAB image is detected, where the a value is used to represent the degree of color from red to yellow, so as to determine the feature color dimension information corresponding to each frame of LAB image.
Then, change information of color contrast between the LAB images of each frame is determined according to the characteristic color dimension information, and the bleeding severity is determined according to the change information of color contrast between the LAB images. The color change degree of the image in the red channel can be more obviously determined in the LAB color space, so that the accuracy of the bleeding severity degree determined based on the color contrast information between the LAB images is improved, the LAB image is obtained by performing color gamut conversion on the first target image through the embodiment, the bleeding severity degree is determined based on the LAB image, and the accuracy of bleeding detection is effectively improved.
In some embodiments, determining the bleeding severity of the target tissue according to the color contrast variation information between the at least two first target images includes:
Step S141, according to the color contrast of the first image to be detected and the second target image in the plurality of frames of second images to be detected, generating first color change trend information corresponding to the second target image.
The second target image may be one of the first target images, or the second target image may be a frame image different from the first target image, which is determined from the first image to be detected and the plurality of frames of second images to be detected. In another embodiment, the second target image is the latest frame image of the first to-be-detected image and the plurality of frames of second to-be-detected images, so as to improve accuracy of a fine detection process for the bleeding condition of the target tissue based on the second target image.
And then, according to the correlation statistics between each pixel point in the whole image of the second target image and the color contrast of each pixel point, determining trend information of the color contrast to obtain the first color change trend information, namely, the first color change trend information is used for indicating the trend information of the color contrast of all the pixel points in the whole second target image. The first color change trend information may be determined by histogram statistics, fitting a curve or fitting a formula. For example, a histogram of color contrast of all pixels in the second target image may be counted, and based on the histogram, trend information of color contrast along with the change of the pixels in a certain color channel may be obtained, so as to obtain the first color change trend information.
Step S142, determining bleeding amount information of the target tissue according to the first color change trend information.
Because the first color change trend information carries the change trend of the color contrast of each pixel point in the whole second target image, the concentration degree of the color contrast of each pixel point in the red chromaticity region can be determined based on the first color change trend, and the occupied area of blood in the second target image at the corresponding moment can be calculated. Specifically, a first mapping relation between the color contrast value and the occupied area of blood in the image can be preset according to historical data or priori knowledge, and the actual occupied area of blood in the image can be determined, for example, when the degree of concentration of the color contrast obtained through statistics accounts for 60% of all the color contrast values, the corresponding occupied area of the actual blood in the image is found to be 10% based on the first mapping relation.
Then, the bleeding amount information is determined according to the occupation area of the detected blood in the image, namely, the larger the occupation area of the blood in the image is, the larger the bleeding amount of the bleeding point of the target tissue is correspondingly. In the present embodiment, the color contrast of the image is counted by the above-described step to obtain a result of quantifying the image area occupied by blood, and thus the corresponding bleeding amount can be quantitatively obtained, compared with the bleeding amount obtained by the qualitative analysis in the above-described step S120. For example, a second mapping relation between the occupied area of blood in the image and the bleeding amount can be preset according to historical data or priori knowledge, and an actual bleeding amount value is determined, for example, when the occupied area of blood in the image is determined to be 10%, the corresponding actual bleeding amount value is found to be 5ml through the second mapping relation. In another embodiment, a mapping relationship between the degree of color contrast concentration and the bleeding amount value may be determined based on the first mapping relationship and the second mapping relationship, and the actual bleeding amount may be calculated based on the mapping relationship.
Step S143, determining the bleeding severity of the target tissue according to the bleeding amount information and the change information of the color contrast between the at least two first target images.
The higher the bleeding amount of the target tissue, i.e. the larger the blood occupied in the image, or the faster the color contrast between the multiple frames of the first target image, i.e. the faster the blood expansion rate in the image, the higher the bleeding severity of the target tissue at this time. Therefore, the bleeding severity can be comprehensively analyzed based on the bleeding amount detection result and the change information of the color contrast between the frame images. Illustratively, when the actual bleeding amount value is in the range of 30ml to 50ml, the change information of the color contrast between frames indicates that the bleeding speed is in the range of A1 ml/min to A2 ml/min, the corresponding bleeding severity is a first-order severity indicating that the bleeding amount of the target tissue is large, the bleeding speed is high, that is, the bleeding severity, and when the actual bleeding amount value is in the range of 15ml to 30ml, the change information of the color contrast between frames indicates that the bleeding speed is in the range of A3ml/min to A1 ml/min, the corresponding bleeding severity is a second-order severity indicating that the bleeding amount of the target tissue is large, the bleeding speed is general.
Through the embodiment, the bleeding amount is further precisely detected through the change information of the color contrast of the second target image in the first image to be detected and the second image to be detected, so that a more accurate bleeding amount detection result can be obtained, and the bleeding detection accuracy is effectively improved.
In an exemplary embodiment, the determining the bleeding amount information of the target tissue according to the first color change trend information further includes the steps of:
And determining a first color contrast to be detected corresponding to the number of the target pixel points in the second target image according to the first color change trend information. And counting the number of pixel points in the same color contrast in the second target image according to the first color change trend information. Since the larger the number of pixels, the more pixels of the image are the color contrast, that is, the larger the area occupied by the color corresponding to the color contrast in the second target image. Therefore, the number of pixels with the numerical value arranged in the first few bits in all the pixel numbers can be used as the number of the target pixels, and the first color contrast to be detected corresponding to the number of the target pixels can be preferentially detected.
And then, under the condition that the first to-be-detected color contrast is detected to be in a preset color area range, calculating first concentration degree information among all the first to-be-detected color contrast, and determining bleeding amount information of the target tissue according to the first concentration degree information. The color region range refers to a color value range corresponding to hemoglobin, which is determined according to a color gamut of the current image. When the first to-be-detected color contrast is detected to be in the color region range, the image region with the highest color contrast in the current image is the region filled with blood, and the first concentration degree information between the first to-be-detected color contrast can be further detected, so that the quantification result of the bleeding area of the target tissue is obtained through calculation.
Specifically, taking the color gamut of the second target image as the LAB color space as an example, fig. 2 is a schematic diagram of pixel statistics of the first target image or the second target image according to an embodiment of the present application. As shown in fig. 2, a fitting curve for representing the trend of the first color change is obtained according to the pixel values of each point in the second target image and the statistics of the value a corresponding to the pixel values of each point, wherein a represents the degree of the color from red to yellow, and the peak in the fitting curve is used for representing the number of the pixel points with the largest numerical value under the same color contrast in the second target image, which indicates that the color represented by the color contrast at the peak occupies the largest area in the image. Since hemoglobin is red, when the chromaticity value of the color contrast at the peak is detected to be within the range of the color region, that is, the range of the red chromaticity value, the first integration degree information of the first color contrast to be detected can be determined according to the value of the peak width Δa near the peak. When the Δa value is smaller, the integration degree of the first color contrast to be detected is higher, that is, the color contrast of more pixel points is closer to the first color contrast to be detected, the occupied area of blood in the second target image is higher, the actual bleeding amount is higher, and further, a specific actual bleeding amount value can be determined based on the Δa value obtained through statistics through the association relation. It will be appreciated that the peak width Δa may be calculated from the peak width at 1/n peak, and n is a positive number greater than 1.
In some embodiments, the determining the bleeding severity of the target tissue according to the transformation information of the color contrast between the at least two frames of the first target image further includes generating second color change trend information corresponding to the first target image according to the color contrast of the first target image of each frame. The trend information of the color contrast is statistically determined according to the association relation between each pixel point in the whole image of each frame of the first target image and the color contrast of each pixel point, and second color change trend information corresponding to each frame of the first target image is obtained, namely, the second color change trend information is used for indicating the trend information of the color contrast of all the pixel points in the whole first target image.
And then, according to the change information among all the second color change trend information, determining the change information of the color contrast, and according to the change information of the color contrast, determining the bleeding speed information of the target tissue, and according to the bleeding speed information, determining the bleeding severity degree of the target tissue. Specifically, the faster the change between the second color change trend information of the first target image of each frame, the faster the blood spreading speed of the bleeding point of the target tissue in the period of time, and the faster the bleeding speed of the target tissue.
In an exemplary embodiment, the determining the change information of the color contrast according to the change information between all the second color change trend information further includes determining, according to the second color change trend information, a second color contrast to be detected corresponding to a target pixel number in the first target image of each frame, where the target pixel number in the first target image may be a pixel number with a numerical rank of a first few bits in all the pixel numbers of the first target image. And under the condition that the second color contrast to be detected is detected to be in a preset color area range, calculating second concentration degree information among all the second color contrast to be detected in each frame of the first target image, and determining the change information of the color contrast according to the change information among the second concentration degree information corresponding to each frame of the first target image.
Specifically, referring to fig. 2, a fitting curve for representing a second color variation trend is obtained according to statistics of pixel values of each point in the first target image and a value a corresponding to the pixel values of each point, and a peak in the fitting curve is used for representing the number of pixel points with the largest value under the same color contrast in the first target image. When the chromaticity value of the color contrast at the peak is within the color area, and the change speed of the peak width deltaa value at the 1/n peak between the first target images of each frame is greater, that is, the change speed between the second integration degree information corresponding to the first target image of each frame is greater, the bleeding speed in the picture is higher and faster, and the change rate information can be determined according to the association relation. It should be noted that, when the change amount of Δa between frames reaches the preset width change amount threshold, the image processing device may determine that the cleaning hemostasis operation is necessary, and alert the operator through display of a display screen or voice prompt.
Through the embodiment, the change rate information between each frame is obtained through calculation by monitoring the second color change trend information of the first target image, so that the bleeding severity degree can be determined through specific calculation values, the quantitative detection method for the bleeding speed of the target tissue is realized, and the accuracy of bleeding detection is improved.
In some embodiments, the image capturing mode of the endoscope includes a white light mode and a non-white light mode, and the bleeding detection method further includes the steps of:
Step S151, under the condition that the bleeding severity is the target severity, switching the image acquisition mode from the white light mode to the non-white light mode, and acquiring a non-white light image of the endoscope in the non-white light mode, wherein the light irradiated by the endoscope to the target tissue in the white light mode is white light, and the light irradiated to the target tissue in the non-white light mode is non-white light.
The method comprises the steps of setting a bleeding speed threshold value or a bleeding amount threshold value and other different preset threshold values, and dividing the bleeding severity of the target tissue, wherein when the embodiment detects that the bleeding amount corresponding to the target image is large and/or the bleeding speed is high, the current target tissue is seriously bleeding, namely the target severity is determined. Conventionally, the endoscope generally acquires images in a white light mode, and acquires the first image to be detected and the second image to be detected. When the bleeding severity of the target tissue is determined to be the target severity, in order for an operator to stop bleeding in time, the image processing device can control the light source device of the endoscope to switch from white light to non-white light, so as to acquire a non-white light image acquired in a non-white light mode at the current moment to detect the bleeding point position. It is understood that, in the above-mentioned non-white light mode, the wavelength band of the illumination light to which the light source device is switched may be generally set to, for example, a near infrared wavelength range of 600nm to 630 nm.
It should be noted that, when the bleeding condition of the target tissue is detected to be at the target severity, the switching mode of the image acquisition mode for the endoscope may include a manual switching method and an automatic switching mode. For example, when the measured values reach the preset threshold value in real time, the display interface can be used for carrying out text information or voice prompt so as to remind an operator of cleaning blood stains in a picture and stopping bleeding as soon as possible, and the operator can manually switch the illumination light source equipment of the endoscope to the non-white light source equipment to realize the switching of the image acquisition mode, or when the current severe bleeding is monitored, namely the image processing equipment can be used for automatically switching the image acquisition mode to control the endoscope to irradiate the non-white light to the target tissue so as to acquire the non-white light image.
Step S152, generating a bleeding position detection result for the target tissue according to the non-white light image.
The bleeding position detection mode based on the non-white light image can adopt modes such as manual detection or algorithm automatic detection. For example, after the endoscope collects the non-white light image, the image processing device may send the collected non-white light image to the display, and the operator may manually detect according to the non-white light image displayed on the display in a current switching manner, so as to determine the specific position of the bleeding point of the target tissue at the current moment, and obtain the bleeding position detection result.
Or the image processing equipment can directly utilize the absorption rate characteristic of hemoglobin to the non-white light of the wave band at the rear end, and the automatic bleeding point position detection is carried out through an algorithm, so that the operation steps required by operators are reduced, and the operation efficiency is improved. Specifically, when the blood area and the area expansion speed in the image reach the threshold value, the voice or the interface prompts an operator to clean blood stains, meanwhile, the bleeding position is searched by a method of switching the image observation mode back and forth at intervals, and the display image is still a white light image, namely, the former frame is an image collected in the white light mode, and the latter frame is an image collected in the non-white light mode, wherein the white light image is displayed in the display. Then, the background automatically determines the bleeding position by a pattern matching method, namely, the blood color in a special pattern is identified by an orange area in a color space, the position closest to red is found in the area in an image, and the specific bleeding position is determined, or the bleeding point is automatically found by a neural network, namely, the neural network trained by the bleeding point identification is used, the non-white light image is input into the neural network, and finally the bleeding point position is output. Or the automatic detection of the specific position of the bleeding point of the target tissue in the background is realized through other target detection algorithms, and the details are not repeated here.
It should be noted that, when the position of the bleeding point is detected through the steps and the operator performs the operation of cleaning and stopping bleeding on the bleeding point of the target tissue, the first target image and/or the second target image are continuously obtained and monitored in real time, and when the proportion of the area of the blood to the screen area at the current moment is monitored to be lower than the threshold value, the image acquisition mode of the endoscope is switched from the non-white light mode to the white light mode by means of manual control or automatic control and the like, and the acquisition work and the real-time monitoring of the endoscope are continuously performed.
Through the steps S151 to S152, when the bleeding severity of the target tissue is detected to be the target severity, the image acquisition mode of the endoscope is switched from the white light mode to the non-white light mode, and the bleeding point is searched by utilizing the absorptivity characteristic of the red protein to the light of the wave band through the acquired non-white light image, so that the accuracy and the efficiency of detecting the bleeding point are improved, and the accuracy and the efficiency of detecting the bleeding point are further improved effectively.
In some embodiments, after determining the bleeding severity of the target tissue, the bleeding detection method further includes generating bleeding position marker information for the target tissue based on the bleeding position detection result, and generating an image to be displayed based on the bleeding position marker information and the white light image.
Specifically, after the detection result of the bleeding position of the target tissue is obtained through the above method embodiment, the specific bleeding position determined based on the non-white light image may be generated into corresponding bleeding position mark information, where the bleeding position mark information may be information for prompting the position of the bleeding point in the image, such as a prompt box or text identification information. And the bleeding position found on the previous frame of non-white light image is almost coincident with the bleeding position on the next frame of white light image without worrying about the bleeding position error of prompt because the interval time between each frame is extremely short and the positions on the front and rear frame images of the same part are not greatly different. And finally, the image to be displayed can be directly sent to a display for interface display, or the image to be displayed is stored in the image processing equipment, and when an operator inputs an interactive display instruction through the display, the stored image to be displayed is sent to the display for display in response to the detected display instruction. Taking a prompt box as an example for bleeding position marking information, in an image to be displayed, according to the phenomenon that the brightness of the image is darker when the image is closer to the center of a bleeding point, an operator can be further helped to position the bleeding point as accurately as possible by changing the brightness in the prompt box in the image, and the bleeding point is stopped by using electric coagulation, so that the visual focus is conveniently focused.
Therefore, through the embodiment, the display picture can continuously keep displaying the white light image acquired in the white light mode, meanwhile, the bleeding position mark information detected by the background based on the non-white light image is superimposed to the white light image, so that the specific position of a bleeding point of an operator is prompted through the white light image, the displayed picture color accords with the color in an organ cavity observed by a human body under normal white light, the operation of the operator is prevented from being influenced, and the accuracy and the efficiency of bleeding detection are effectively improved.
The following description is made with reference to specific embodiments. Fig. 3 is a flowchart of a bleeding detection method according to a preferred embodiment of the present application, and as shown in fig. 3, the calibration procedure includes the steps of:
in step S301, a first image to be detected and a second image to be detected acquired by an endoscope are monitored.
Step S302, judging whether the image monitoring result obtained in the step reaches a threshold value, if so, executing the subsequent step S303, and if not, continuing to execute the step S301.
In step S303, the display performs image or sound prompt, and outputs white light image and non-white light image respectively at intervals of frames.
In step S304, the display displays a white light image, and the background automatically searches for bleeding points from the non-white light image.
Step S305, the operator cleans and stops bleeding points of the target tissue based on the monitoring result.
It should be noted that the steps illustrated in the above-described flow or flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order other than that illustrated herein.
The present embodiment also provides a bleeding detection system, which is used to implement the foregoing embodiments and preferred embodiments, and is not described in detail. As used below, the terms "portion," "unit," "sub-unit," and the like may be a combination of software and/or hardware that implements a predetermined function. While the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
Fig. 4 is a block diagram of a bleeding detection system according to an embodiment of the present application, and as shown in fig. 4, the system includes a first acquisition portion 42, a first determination portion 44, a second acquisition portion 46, and a second determination portion 48. The first obtaining part 42 is used for obtaining a first to-be-detected image of a target tissue collected by an endoscope, the first determining part 44 is used for determining the bleeding amount condition of the target tissue according to the image gradient of the first to-be-detected image, the second obtaining part 46 is used for obtaining a plurality of frames of second to-be-detected images of the target tissue collected by the endoscope before and/or after the first to-be-detected image under the condition that the bleeding amount condition of the target tissue is the target bleeding amount, and the second determining part 48 is used for determining the bleeding severity of the target tissue according to the change information of the color contrast between at least two frames of first to-be-detected images.
Through the above embodiment, the first determining part 44 primarily determines whether to detect the bleeding amount condition of the target tissue through the image gradient of the first to-be-detected image, and the second determining part 48 detects the bleeding severity through the change information of the color contrast between the first to-be-detected image and the first to-be-detected image in the plurality of frames of the second to-be-detected images under the condition of determining the target bleeding amount, so that the bleeding condition of the bleeding point can be detected in time when the bleeding point occurs, and the bleeding point can be detected in real time, the phenomenon that the bleeding point cannot be detected in time or the bleeding detection accuracy is low due to the untimely processing of the endoscope acquisition image is avoided, the problem of low instantaneity and accuracy of the bleeding condition detection is solved, and the timely and efficient bleeding point detection system is realized.
In some embodiments, the bleeding detection device further comprises a mode switching part, wherein the mode switching part is used for switching the image acquisition mode from the white light mode to the non-white light mode and acquiring a non-white light image of the endoscope in the non-white light mode when the bleeding severity is the target severity, the light irradiated by the endoscope to the target tissue in the white light mode is white light, the light irradiated by the endoscope to the target tissue in the non-white light mode is non-white light, and the mode switching part generates a bleeding position detection result aiming at the target tissue according to the non-white light image.
In some embodiments, the bleeding detection device further comprises a display part, wherein the display part is used for generating bleeding position mark information aiming at the target tissue based on the bleeding position detection result, and the display part is used for generating an image to be displayed according to the bleeding position mark information and the white light image.
The above-described respective modules may be functional modules or program modules, and may be implemented by software or hardware. For modules implemented in hardware, the modules may be located in the same processor, or may be located in different processors in any combination.
The present embodiment also provides an image processing apparatus comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
Optionally, the image processing apparatus may further include a transmission apparatus connected to the processor, and an input/output apparatus connected to the processor.
Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:
s1, acquiring a first to-be-detected image of target tissue acquired by an endoscope.
S2, determining the bleeding amount of the target tissue according to the image gradient of the first image to be detected.
S3, under the condition that the bleeding amount of the target tissue is the target bleeding amount, acquiring a plurality of frames of second to-be-detected images of the target tissue acquired by the endoscope before and/or after the first to-be-detected images.
And S4, determining the bleeding severity of the target tissue according to the change information of the color contrast between at least two frames of first target images, wherein the first target images are images in the first image to be detected and the plurality of frames of second images to be detected.
It should be noted that, specific examples in this embodiment may refer to examples described in the foregoing embodiments and alternative implementations, and this embodiment is not repeated herein.
This embodiment also provides an endoscope system, and fig. 5 is a schematic view of the structure of an endoscope system according to an embodiment of the present application, and as shown in fig. 5, the system includes a display 52, an endoscope 54, a light source device 56, and an image processing device 58 as described in the above embodiments. In another embodiment, the image processing device 58 may also be deployed in a cloud box connected to the endoscope system via a network.
The present embodiment also provides a computer device, which may be a server device, and fig. 6 is a structural diagram of an inside of the computer device according to an embodiment of the present application, as shown in fig. 6. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing a first image to be detected and a second image to be detected. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of bleeding detection.
It will be appreciated by those skilled in the art that the structure shown in FIG. 6 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In addition, in combination with the bleeding detection method in the above embodiment, the embodiment of the present application may be implemented by providing a storage medium. The storage medium having stored thereon a computer program which, when executed by a processor, implements any of the methods of bleeding detection of the above embodiments.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It should be understood by those skilled in the art that the technical features of the above-described embodiments may be combined in any manner, and for brevity, all of the possible combinations of the technical features of the above-described embodiments are not described, however, they should be considered as being within the scope of the description provided herein, as long as there is no contradiction between the combinations of the technical features.
The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.