CN115115755B

Movatterモバイル変換

Info

Publication number: CN115115755B
Application number: CN202211043998.4A
Authority: CN
Inventors: 蔡惠明; 李长流; 王子阳; 倪轲娜; 卢露
Original assignee: Nanjing Nuoyuan Medical Devices Co Ltd
Current assignee: Nanjing Nuoyuan Medical Devices Co Ltd
Priority date: 2022-08-30
Filing date: 2022-08-30
Publication date: 2022-11-08
Anticipated expiration: 2042-08-30
Also published as: CN115115755A; WO2024045507A1

Abstract

Description

Fluorescence three-dimensional imaging method and device based on data processing

Technical Field

The invention relates to a data processing technology, in particular to a fluorescence three-dimensional imaging method and device based on data processing.

Background

The fluorescence three-dimensional imaging is that the intensity of a fluorescence signal emitted after a fluorescent substance is excited is in a linear relation with the amount of fluorescein in a certain range.

At present, after imaging is performed by a fluorescence three-dimensional imaging device, only a corresponding fluorescence three-dimensional image can be obtained, a user can only perform integral amplification and reduction observation on a lesion area according to the obtained fluorescence three-dimensional image, and cannot observe an amplified image of a part of tissues of the lesion area and a white light image and a fluorescence image corresponding to a part of tissues at the same time on a display screen, so that the diagnosis efficiency of a doctor is low, and the use of the doctor is inconvenient.

Disclosure of Invention

The embodiment of the invention provides a fluorescence three-dimensional imaging method and device based on data processing, wherein a fluorescence image is combined with a white light image, so that a fluorescence area of a pathological change part can be directly displayed in the white light image, the observation of a doctor is facilitated, the detail display can be carried out on the area selected by the doctor and the related area can be displayed, even the part selected by the doctor and the related area are spliced and combined to automatically push the complete pathological change part, the observation of the doctor is facilitated, and the diagnosis result is more accurate.

In a first aspect of the embodiments of the present invention, a fluorescence three-dimensional imaging method based on data processing is provided, including:

performing image extraction on the pathological tissue under fluorescence and white light based on a two-dimensional image extraction device to obtain fluorescence two-dimensional images and white light two-dimensional images at different region positions of the pathological tissue, and generating a fluorescence three-dimensional fusion image corresponding to the pathological tissue according to the fluorescence two-dimensional images and the white light two-dimensional images;

performing block division on the fluorescence three-dimensional fusion image according to the region position to obtain a plurality of sub three-dimensional images, establishing a preset storage space corresponding to each sub three-dimensional image, and storing the fluorescence two-dimensional image and the white light two-dimensional image of the corresponding region position in the preset storage space;

displaying the fluorescent three-dimensional fusion image, and calling a first fluorescent two-dimensional image and a first white light two-dimensional image corresponding to a first sub three-dimensional image for displaying if the first sub three-dimensional image is judged to be selected by a user;

determining a plurality of second sub three-dimensional images according to the region positions of the first sub three-dimensional images, and determining at least one third fluorescent two-dimensional image according to the number of fluorescent pixel points in the second fluorescent two-dimensional image corresponding to each second sub three-dimensional image, wherein the fluorescent pixel points are pixel points in a preset fluorescent pixel value interval;

and generating an image display frame according to the number of the third fluorescent two-dimensional images, and displaying the fluorescent three-dimensional fusion image, the first fluorescent two-dimensional image, the first white light two-dimensional image, the third fluorescent two-dimensional image and the third white light two-dimensional image corresponding to the third fluorescent two-dimensional image on the basis of the image display frame.

Optionally, in a possible implementation manner of the first aspect, performing image extraction on a lesion tissue under fluorescence and white light based on a two-dimensional image extraction device to obtain a fluorescence two-dimensional image and a white light two-dimensional image at different region positions of the lesion tissue, and generating a fluorescence three-dimensional fusion image corresponding to the lesion tissue according to the fluorescence two-dimensional image and the white light two-dimensional image includes:

selecting any pixel point at the same position in the fluorescent two-dimensional image and the white light two-dimensional image as a first coordinate center point;

establishing coordinate systems at the fluorescent two-dimensional image and the white light two-dimensional image respectively according to the first coordinate center point to obtain a first fluorescent set and a first white light set;

extracting pixel points in a preset fluorescence pixel value interval in the first fluorescence set to obtain a target pixel point set;

replacing the pixel value of each pixel point in the target pixel point set with the pixel value of a pixel point with the same coordinate value in the first white light set to obtain a fluorescent two-dimensional fusion image;

and sending the fluorescence two-dimensional fusion image to a server for splicing and combining to obtain a fluorescence three-dimensional fusion image.

Optionally, in a possible implementation manner of the first aspect, the performing block division on the fluorescence three-dimensional fusion image according to the region position to obtain a plurality of sub three-dimensional images, establishing a preset storage space corresponding to each sub three-dimensional image, and storing the fluorescence two-dimensional image and the white light two-dimensional image at the corresponding region position in the preset storage space includes:

obtaining a corresponding actual division size according to the actual size of the fluorescence three-dimensional fusion image;

sequentially carrying out block division on the fluorescence three-dimensional fusion image from top to bottom according to the actual division size to obtain a plurality of sub three-dimensional images until the fluorescence three-dimensional fusion image is divided into the sub three-dimensional images, and sequentially numbering according to the division sequence;

and establishing a preset storage space corresponding to each number, and storing the fluorescent two-dimensional image and the white light two-dimensional image of the corresponding region position in the preset storage space.

Optionally, in a possible implementation manner of the first aspect, the obtaining a corresponding actual division size according to an actual size of the fluorescence three-dimensional fusion image includes:

obtaining a size coefficient according to the ratio of the actual size of the fluorescence three-dimensional fusion image to the preset size of the fluorescence three-dimensional fusion image;

obtaining an actual division size according to the product of the size coefficient and a preset division size;

the actual division size is obtained by the following formula,

wherein,

in order to actually divide the size of the wafer,

in order to preset the division size,

is the actual size of the fluorescence three-dimensional fusion image,

is a preset size of the fluorescence three-dimensional fusion image,

is an attribute value of the actual division size.

Optionally, in a possible implementation manner of the first aspect, the determining, according to the area position of the first sub three-dimensional image, a plurality of second sub three-dimensional images, and determining, according to the number of fluorescence pixel points in the second fluorescence two-dimensional image corresponding to each second sub three-dimensional image, at least one third fluorescence two-dimensional image includes:

determining a region position adjacent to the first sub three-dimensional image, and taking an image of the region position as a second sub three-dimensional image;

determining pixel points adjacent to the second fluorescent two-dimensional image in the first fluorescent two-dimensional image as boundary fluorescent pixel points;

counting the number of the fluorescence pixel points which are directly connected and indirectly connected with the boundary fluorescence pixel points in the second fluorescence two-dimensional image to obtain a first total number;

and if the first total number is larger than the preset number, taking the second fluorescent two-dimensional image as a third fluorescent two-dimensional image.

Optionally, in a possible implementation manner of the first aspect, the counting the number of fluorescence pixel points in the second fluorescence two-dimensional image that are directly connected and indirectly connected to the boundary fluorescence pixel point to obtain a first total number includes:

determining fluorescent pixel points adjacent to the boundary fluorescent pixel points in the second fluorescent two-dimensional image to obtain a plurality of first pixel points;

determining fluorescent pixel points adjacent to the first pixel points to obtain a plurality of second pixel points, determining fluorescent pixel points adjacent to the second pixel points to obtain a plurality of third pixel points, taking the third pixel points as the second pixel points, and determining the third pixel points connected with the second pixel points again until all the second pixel points are selected;

the first pixel point is directly connected with the boundary fluorescent pixel point, and the second pixel point is indirectly connected with the boundary fluorescent pixel point;

and counting the number of the first pixel points and the second pixel points to obtain a first total number.

Optionally, in a possible implementation manner of the first aspect, the generating an image display frame according to the number of the third fluorescent two-dimensional images, and displaying the fluorescent three-dimensional fusion image, the first fluorescent two-dimensional image, the first white-light two-dimensional image, the third fluorescent two-dimensional image, and a third white-light two-dimensional image corresponding to the third fluorescent two-dimensional image based on the image display frame includes:

determining a three-dimensional image display area and a two-dimensional image display area in the initialized image display frame;

dividing the two-dimensional image display area according to the number of the third fluorescent two-dimensional images to obtain a first two-dimensional display sub-area and a second two-dimensional display sub-area;

respectively establishing a white light display area and a fluorescent display area in the first two-dimensional display sub-area and the second two-dimensional display sub-area;

and displaying the first fluorescence two-dimensional image and the first white light two-dimensional image in a white light display area and a fluorescence display area of the first two-dimensional display sub-area, and sequentially displaying the third fluorescence two-dimensional image and the third white light two-dimensional image in the first fluorescence sequence in the white light display area and the fluorescence display area of the second two-dimensional display sub-area.

Optionally, in a possible implementation manner of the first aspect, the method further includes:

and if the number of the third fluorescent two-dimensional images is more than one, sequencing the third fluorescent two-dimensional images in a descending order according to the first total number to obtain a first fluorescent sequence.

selecting fluorescent pixel points adjacent to the non-fluorescent pixel points in the first fluorescent two-dimensional image to obtain a first contour fluorescent pixel point set;

selecting fluorescent pixel points adjacent to the non-fluorescent pixel points in the third fluorescent two-dimensional image to obtain at least one second contour fluorescent pixel point set;

the first contour fluorescence pixel point set and the second contour fluorescence pixel point set are spliced based on the positions of the first fluorescence two-dimensional image and the third fluorescence two-dimensional image to obtain a target contour fluorescence two-dimensional image;

and establishing a third two-dimensional display sub-area in the two-dimensional image display area, wherein the third two-dimensional display sub-area is used for displaying a fluorescence two-dimensional image of the target contour.

In a second aspect of the embodiments of the present invention, there is provided a fluorescence three-dimensional imaging apparatus based on data processing, including:

the fusion module is used for extracting images of the lesion tissue under fluorescence and white light based on the two-dimensional image extraction device to obtain fluorescence two-dimensional images and white light two-dimensional images of different region positions of the lesion tissue, and generating a fluorescence three-dimensional fusion image corresponding to the lesion tissue according to the fluorescence two-dimensional images and the white light two-dimensional images;

the division module is used for carrying out block division on the fluorescent three-dimensional fusion image according to the region position to obtain a plurality of sub three-dimensional images, establishing a preset storage space corresponding to each sub three-dimensional image, and storing the fluorescent two-dimensional image and the white light two-dimensional image of the corresponding region position in the preset storage space;

the calling module is used for displaying the fluorescent three-dimensional fusion image, and calling a first fluorescent two-dimensional image and a first white light two-dimensional image corresponding to a first sub three-dimensional image for displaying if the fact that the user selects the first sub three-dimensional image is judged;

the determining module is used for determining a plurality of second sub three-dimensional images according to the region positions of the first sub three-dimensional images, and determining at least one third fluorescent two-dimensional image according to the number of fluorescent pixel points in the second fluorescent two-dimensional image corresponding to each second sub three-dimensional image, wherein the fluorescent pixel points are pixel points in a preset fluorescent pixel value interval;

and the display module is used for generating an image display frame according to the number of the third fluorescent two-dimensional images and displaying the fluorescent three-dimensional fusion image, the first fluorescent two-dimensional image, the first white light two-dimensional image, the third fluorescent two-dimensional image and the third white light two-dimensional image corresponding to the third fluorescent two-dimensional image on the basis of the image display frame.

In a third aspect of the embodiments of the present invention, there is provided an electronic device, including: memory, a processor and a computer program, the computer program being stored in the memory, the processor running the computer program to perform the method of the first aspect of the invention as well as various possible aspects of the first aspect.

A fourth aspect of the embodiments of the present invention provides a readable storage medium, in which a computer program is stored, and the computer program is used for implementing the first aspect of the present invention and various possible related methods of the first aspect when executed by a processor.

The invention provides a fluorescence three-dimensional imaging method and a fluorescence three-dimensional imaging device based on data processing, which can fuse and splice two-dimensional images of pathological change tissues under the irradiation of fluorescence and white light respectively to obtain a fluorescence three-dimensional fused image, so that a doctor can directly observe a pathological change region in the white light image, the doctor can conveniently observe the pathological change region, and can judge the degree of cytopathic change by directly using different colors of fluorescence pixel values in the white light image, and the doctor can conveniently perform subsequent operations.

According to the technical scheme provided by the invention, the divided regions with different sizes are generated for the lesion regions with different sizes, so that the sizes of the subsequently selected and automatically pushed observation regions are proper, and the observation regions are more beneficial to the observation of doctors.

According to the technical scheme provided by the invention, the image display frame is divided into the three-dimensional display area and the two-dimensional display area, the three-dimensional display area is used for displaying the fluorescent three-dimensional fusion image, the two-dimensional display area is respectively used for displaying the fluorescent and white light image of the area selected by a doctor and the fluorescent and white light image of the area associated with the lesion area in the area selected by the doctor, the fluorescent and white light images are displayed in sequence according to the severity of the associated area, and the selected area and the outline of the associated area are spliced to form a complete lesion part for displaying, so that the doctor can observe the whole lesion part, can observe the selected partial lesion area, the associated area and the corresponding spliced partial whole, can observe the doctor at multiple angles, and can relatively objectively and accurately diagnose.

Drawings

FIG. 1 is a flow chart of a fluorescence three-dimensional imaging method based on data processing according to the present invention;

FIG. 2 is a flow chart of a method for locating a third fluorescence two-dimensional image according to the present invention;

FIG. 3 is a schematic structural diagram of a fluorescence three-dimensional imaging device based on data processing according to the present invention;

fig. 4 is a schematic diagram of a hardware structure of an electronic device according to the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein.

It should be understood that, in various embodiments of the present invention, the sequence numbers of the processes do not mean the execution sequence, and the execution sequence of the processes should be determined by the functions and the internal logic of the processes, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

It should be understood that in the present application, "comprising" and "having" and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that, in the present invention, "a plurality" means two or more. "and/or" is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "comprising a, B and C", "comprising a, B, C" means that all three of a, B, C are comprised, "comprising a, B or C" means comprising one of a, B, C, "comprising a, B and/or C" means comprising any 1 or any 2 or 3 of a, B, C.

It should be understood that in the present invention, "B corresponding to a", "a corresponds to B", or "B corresponds to a" means that B is associated with a, and B can be determined from a. Determining B from a does not mean determining B from a alone, but may be determined from a and/or other information. And the matching of A and B means that the similarity of A and B is greater than or equal to a preset threshold value.

As used herein, the term "if" may be interpreted as "at \8230; …" or "in response to a determination" or "in response to a detection" depending on the context.

The technical solution of the present invention will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.

The invention provides a fluorescence three-dimensional imaging method based on data processing, as shown in figure 1, comprising the following steps:

and S110, extracting images of the lesion tissue under fluorescence and white light based on a two-dimensional image extraction device to obtain fluorescence two-dimensional images and white light two-dimensional images of different region positions of the lesion tissue, and generating and processing the fluorescence two-dimensional images and the white light two-dimensional images.

According to the technical scheme provided by the invention, the two-dimensional image extraction device controls incident light to perform annular scanning at the same incident angle and different direction angles of 360 degrees, performs image extraction on the lesion tissue under fluorescence and white light respectively to obtain fluorescence two-dimensional images and white light two-dimensional images at different region positions of the lesion tissue, sends all the fluorescence two-dimensional images and the white light two-dimensional images to the server for splicing treatment, and waits for a fluorescence three-dimensional fusion image corresponding to the lesion tissue.

In a possible implementation manner of the technical solution provided by the present invention, step S110 specifically includes:

and selecting any pixel point at the same position in the fluorescent two-dimensional image and the white light two-dimensional image as a first coordinate center point.

According to the technical scheme provided by the invention, the system can select any one pixel point at the same position in the fluorescent two-dimensional image and the white light two-dimensional image as the first coordinate center point, and it can be understood that the fluorescent two-dimensional image and the white light two-dimensional image in each area are at the same angle, and are shot at the same distance, and only the irradiated light sources are different, so that the sizes of the fluorescent two-dimensional image and the white light two-dimensional image are the same and are mutually corresponding, therefore, any one pixel point at the same position in the fluorescent two-dimensional image and the white light two-dimensional image is selected as the first coordinate center point, and a coordinate system is conveniently established according to the first coordinate center point to obtain a pixel point set of the two-dimensional images.

And establishing coordinate systems at the fluorescent two-dimensional image and the white light two-dimensional image respectively according to the first coordinate center point to obtain a first fluorescent set and a first white light set.

According to the technical scheme provided by the invention, the coordinate systems are respectively established at the fluorescent two-dimensional image and the white light two-dimensional image according to the first coordinate center point, the fluorescent two-dimensional image and the white light two-dimensional image are decomposed according to the coordinate systems to obtain the first fluorescent set and the first white light set, so that the fluorescent pixel points of the focus region in the fluorescent two-dimensional image can be conveniently replaced into the white light two-dimensional image in the follow-up process, and a doctor can conveniently and directly observe the position and the corresponding severity of the focus region.

And extracting pixel points in a preset pixel interval in the first fluorescence set to obtain a target pixel point set.

According to the technical scheme provided by the invention, the system extracts the pixel points in the preset pixel interval in the first fluorescence set to obtain the corresponding target pixel point set, and the preset pixel interval is the pixel value interval of the fluorescence pixel points, all the fluorescence points in the fluorescence two-dimensional image are extracted, the fluorescence pixel points are focus positions, the target pixel point set is a focus area, the fluorescence pixel points are conveniently replaced to the area corresponding to the white light two-dimensional image in the follow-up process, and the check of a doctor is facilitated.

And replacing the pixel value of each pixel point in the target pixel point set with the pixel value of a pixel point with the same coordinate value in the first white light set to obtain a fluorescent two-dimensional fusion image.

According to the technical scheme provided by the invention, the pixel value of each pixel point in the target pixel point set is replaced with the pixel value of the pixel point with the same coordinate value in the first white light set, and the target pixel point set is understood as a focus area, and the focus area is replaced to the white light two-dimensional image, so that the focus area, namely the fluorescent pixel point part, namely the focus area is directly displayed in the white light two-dimensional image, and a doctor can conveniently and directly observe the focus area.

According to the technical scheme, the fluorescence two-dimensional fusion image is sent to a server, and the fluorescence three-dimensional fusion image is generated by splicing according to the shooting sequence of annular scanning.

And S120, performing block division on the fluorescent three-dimensional fusion image according to the region position to obtain a plurality of sub three-dimensional images, establishing a preset storage space corresponding to each sub three-dimensional image, and storing the fluorescent two-dimensional image and the white light two-dimensional image of the corresponding region position in the preset storage space.

According to the technical scheme provided by the invention, the system can perform block division on the fluorescence three-dimensional fusion image according to the area position to correspondingly obtain a plurality of sub three-dimensional images, a preset storage space corresponding to each sub three-dimensional image is established, and the fluorescence two-dimensional image and the white light two-dimensional image at the area position corresponding to the sub three-dimensional image are stored in the preset storage space.

In a possible embodiment of the technical solution provided by the present invention, step S120 specifically includes:

and obtaining a corresponding actual division size according to the actual size of the fluorescent three-dimensional fusion image.

According to the technical scheme provided by the invention, the corresponding actual division size is obtained according to the actual size of the fluorescence three-dimensional fusion image, so that the subsequent image division of the fluorescence three-dimensional fusion image is convenient, and as the actual division size of the fluorescence three-dimensional fusion image is larger, the actual division size of the fluorescence three-dimensional fusion image is smaller, the corresponding actual division sizes of different lesion tissues are different, so that the corresponding sizes are different, and the division under different conditions is carried out according to the lesion areas with different sizes, so that the divided sizes conform to the actual observation conditions, and the corresponding observation of doctors is facilitated.

In a possible embodiment, the obtaining of the corresponding actual division size according to the actual size of the fluorescent three-dimensional fusion image includes:

and obtaining a size coefficient according to the ratio of the actual size of the fluorescence three-dimensional fusion image to the preset size of the fluorescence three-dimensional fusion image.

According to the technical scheme provided by the invention, the system can obtain the corresponding size coefficient according to the ratio of the actual size of the fluorescent three-dimensional fusion image to the preset size of the fluorescent three-dimensional fusion image, so that the corresponding actual division size can be conveniently generated in the subsequent process.

And obtaining the actual division size according to the product of the size coefficient and the preset division size.

According to the technical scheme provided by the invention, the actual division size corresponding to the current fluorescence three-dimensional fusion image is obtained according to the product of the size coefficient and the preset division size.

The actual division size is obtained by the following formula,

wherein,

in order to actually divide the size of the wafer,

in order to preset the division size,

is the actual size of the fluorescence three-dimensional fusion image,

is a preset size of the fluorescence three-dimensional fusion image,

for actual dimensioningThe value of the attribute(s) is,

as a size factor, it can be understood that the actual size and the actual division size of the fluorescence three-dimensional fusion image

Proportional, actual division of the size of the attribute value

May be preset manually.

According to the technical scheme provided by the invention, the actual division sizes with different sizes are correspondingly generated according to the size difference of the fluorescence three-dimensional fusion image, so that the division sizes are dynamically changed, and the division areas are appropriately sized according to the size change of the focus area, thereby being more convenient for doctors to observe and fitting the actual situation.

And sequentially carrying out block division on the fluorescence three-dimensional fusion image from top to bottom according to the actual division size to obtain a plurality of sub three-dimensional images until the fluorescence three-dimensional fusion image is divided into the sub three-dimensional images, and sequentially numbering according to the division sequence.

According to the technical scheme provided by the invention, the fluorescent three-dimensional fusion image is sequentially subjected to block division from top to bottom according to the actual division size obtained by system calculation to obtain a plurality of sub three-dimensional images until the fluorescent three-dimensional fusion image is divided, and the sub three-dimensional images are sequentially numbered according to the division sequence, so that each sub three-dimensional image has a corresponding number, and the corresponding storage space is conveniently distributed according to the numbers in the follow-up process.

According to the technical scheme provided by the invention, the system can establish the preset storage space corresponding to each serial number, and the fluorescent two-dimensional image and the white light two-dimensional image corresponding to the sub three-dimensional image corresponding to each serial number are stored in the preset storage space corresponding to the serial number.

And S130, displaying the fluorescent three-dimensional fusion image, and calling a first fluorescent two-dimensional image and a first white light two-dimensional image corresponding to a first sub three-dimensional image for displaying if the first sub three-dimensional image is selected by a user.

According to the technical scheme provided by the invention, the system can display the fluorescence three-dimensional fusion image, if the first sub three-dimensional image selected by the user is judged, the first fluorescence two-dimensional image and the first white light two-dimensional image corresponding to the first sub three-dimensional image selected by the user are called to be displayed, and a doctor can display a partial region of a lesion region, so that the doctor can conveniently observe a local part finely, and a subsequent diagnosis result is relatively objective.

Step S140, determining a plurality of second sub three-dimensional images according to the region positions of the first sub three-dimensional images, and determining at least one third fluorescent two-dimensional image according to the number of fluorescent pixel points in the second fluorescent two-dimensional image corresponding to each second sub three-dimensional image, wherein the fluorescent pixel points are pixel points in a preset fluorescent pixel value interval.

According to the technical scheme provided by the invention, a plurality of second sub three-dimensional images are determined according to the region position of the first sub three-dimensional image, and it can be understood that other adjacent sub three-dimensional images can be determined according to the position of the first sub three-dimensional image actively selected by a doctor, the second sub three-dimensional images are the sub three-dimensional images adjacent to the first sub three-dimensional image, whether fluorescent pixel points in the second sub three-dimensional images are connected with pixel points on the edge in the first sub three-dimensional image or not is checked, if the connected pixel points appear, the second sub three-dimensional images are taken as third fluorescent two-dimensional images, and it can be understood that the third fluorescent two-dimensional images are fluorescent two-dimensional images related to a lesion region in the first fluorescent two-dimensional images, and automatic pushing can be conveniently carried out according to the region selected by the doctor in the follow-up process.

In a possible embodiment of the technical solution provided by the present invention, as shown in fig. 2, step S140 specifically includes:

step S1401, determining a region position adjacent to the first sub three-dimensional image, and taking an image of the region position as a second sub three-dimensional image.

According to the technical scheme provided by the invention, other adjacent sub three-dimensional images are determined according to the area where the first sub three-dimensional image is actively selected by a doctor, and the adjacent sub three-dimensional image is taken as the second sub three-dimensional image, so that the second sub three-dimensional image is the sub three-dimensional image adjacent to the first sub three-dimensional image, and the image related to the first sub three-dimensional image is conveniently determined according to the peripheral sub three-dimensional image.

Step S1402, determining pixel points adjacent to the second fluorescent two-dimensional image in the first fluorescent two-dimensional image as boundary fluorescent pixel points.

According to the technical scheme provided by the invention, the system determines pixel points adjacent to the second fluorescent two-dimensional image corresponding to the second sub three-dimensional image around the first fluorescent two-dimensional image corresponding to the first sub three-dimensional image as boundary fluorescent pixel points in the first fluorescent two-dimensional image, and the boundary fluorescent pixel points are peripheral (edge) fluorescent pixel points in the first fluorescent two-dimensional image and are adjacent to the peripheral second fluorescent two-dimensional image.

And S1403, counting the number of the fluorescence pixel points which are directly connected and indirectly connected with the boundary fluorescence pixel points in the second fluorescence two-dimensional image to obtain a first total number.

According to the technical scheme provided by the invention, the system can count the number of the fluorescence pixel points which are directly connected and indirectly connected with the corresponding boundary fluorescence pixel point in each second fluorescence two-dimensional image to obtain the corresponding first total number, and it can be understood that the second fluorescence two-dimensional image is adjacent to the first fluorescence two-dimensional image, and the boundary fluorescence pixel points are the fluorescence pixel points around the first fluorescence two-dimensional image, such as: the coordinate of a boundary fluorescence pixel point 1 on the uppermost side of the first fluorescence two-dimensional image is (1, 1), the coordinate of a fluorescence pixel point 2 directly connected with the boundary fluorescence pixel point exists in a second fluorescence two-dimensional image connected with the first fluorescence two-dimensional image on the upper side of the first fluorescence two-dimensional image is (1, 2), and the coordinate of a fluorescence pixel point 3 exists in the second fluorescence two-dimensional image is (1, 3).

In a possible embodiment, the counting the number of the fluorescence pixels directly and indirectly connected to the border fluorescence pixel in the second fluorescence two-dimensional image to obtain a first total number includes:

and determining fluorescent pixel points adjacent to the boundary fluorescent pixel points in the second fluorescent two-dimensional image to obtain a plurality of first pixel points.

According to the technical scheme provided by the invention, the fluorescence pixel points adjacent to the boundary fluorescence pixel points in the second fluorescence two-dimensional image corresponding to the second sub three-dimensional image are determined to obtain a plurality of first pixel points, and it can be understood that the relationship between the first pixel points and the boundary fluorescence pixel points is a direct connection relationship.

And determining the fluorescent pixel points adjacent to the first pixel points to obtain a plurality of second pixel points, determining the fluorescent pixel points adjacent to the second pixel points to obtain a plurality of third pixel points, taking the third pixel points as the second pixel points, and determining the third pixel points connected with the second pixel points again until all the second pixel points are selected completely.

According to the technical scheme, the fluorescent pixel points adjacent to the first pixel points are determined to obtain a plurality of second pixel points, it can be understood that the relationship between the second pixel points and the boundary fluorescent pixel points is an indirect connection relationship, the fluorescent pixel points adjacent to the second pixel points are determined again to obtain a plurality of third pixel points, the relationship between the third pixel points and the boundary fluorescent pixel points is an indirect connection relationship, and then the third pixel points are used as the second pixel points to find the pixel points adjacent to the third pixel points until the second pixel points are selected completely, namely until the indirectly connected pixel points cannot be found.

The first pixel point is directly connected with the boundary fluorescent pixel point, and the second pixel point is indirectly connected with the boundary fluorescent pixel point.

According to the technical scheme provided by the invention, the first pixel point is directly connected with the boundary fluorescent pixel point, the second pixel point is indirectly connected with the boundary fluorescent pixel point, and the first pixel point and the second pixel point are both associated with the lesion area selected by a doctor, so that the automatic pushing and displaying of the related images can be conveniently carried out subsequently according to the number of the first pixel point and the second pixel point.

According to the technical scheme provided by the invention, the system can count the number of the first pixel points and the second pixel points to obtain the first total number, and the first total number is understood to be the number of the fluorescent pixel points which are directly and indirectly connected with the boundary fluorescent pixel points in the second fluorescent two-dimensional image.

And step S1404, if the first total number is larger than a preset number, taking the second fluorescence two-dimensional image as a third fluorescence two-dimensional image.

According to the technical scheme provided by the invention, if the first total number corresponding to the second fluorescence two-dimensional image is greater than the preset number, the second sub three-dimensional image is used as the third fluorescence two-dimensional image, it can be understood that the preset number can be 0, when directly connected and/or indirectly connected pixel points exist, the second fluorescence two-dimensional image is determined to be related to a lesion area selected by a doctor, and the second sub three-dimensional image is used as the third fluorescence two-dimensional image, so that the subsequent automatic pushing is convenient.

And S150, generating an image display frame according to the number of the third fluorescent two-dimensional images, and displaying the fluorescent three-dimensional fusion image, the first fluorescent two-dimensional image, the first white light two-dimensional image, the third fluorescent two-dimensional image and the third white light two-dimensional image corresponding to the third fluorescent two-dimensional image on the basis of the image display frame.

According to the technical scheme, the initialization image display frame is divided according to the number of the third fluorescent two-dimensional images to generate the image display frame, the larger the number of the third fluorescent two-dimensional images is relative to the display area to be displayed, the more the number of the third fluorescent two-dimensional images is, the more the image display frame displays the fluorescent three-dimensional fusion image, the first fluorescent two-dimensional image, the first white light two-dimensional image, the third fluorescent two-dimensional image and the third white light two-dimensional image corresponding to the third fluorescent two-dimensional image.

In a possible implementation manner of the technical solution provided by the present invention, step S150 specifically includes:

and determining a three-dimensional image display area and a two-dimensional image display area in the initialized image display frame.

According to the technical scheme provided by the invention, the three-dimensional image display area and the two-dimensional image display area are determined in the initialized image display frame, wherein the initialized image display frame is a blank display frame, and the display area is not divided.

And dividing the two-dimensional image display area according to the number of the third fluorescent two-dimensional images to obtain a first two-dimensional display sub-area and a second two-dimensional display sub-area.

According to the technical scheme provided by the invention, the system can divide the two-dimensional image display area according to the number of the third fluorescent two-dimensional images, and can be understood that the number of the third fluorescent two-dimensional images is 1, the two-dimensional image display area is divided once to obtain the first two-dimensional display sub-area and the second two-dimensional display sub-area, the first two-dimensional display sub-area is used for displaying the first fluorescent two-dimensional image and the first white light two-dimensional image corresponding to the first sub three-dimensional image, the second two-dimensional display sub-area is used for displaying the third fluorescent two-dimensional image and the third white light two-dimensional image corresponding to the third sub three-dimensional image, a doctor can observe the fluorescent three-dimensional fusion image and the image of the selected area at the same time, and the system can also automatically push the two-dimensional images related to the selected area to display a doctor, so that the diagnosis of the doctor is more accurate.

And respectively establishing a white light display area and a fluorescent display area in the first two-dimensional display sub-area and the second two-dimensional display sub-area.

According to the technical scheme provided by the invention, the white light display area and the fluorescence display area are respectively established in the first two-dimensional display sub-area and the second two-dimensional display sub-area, and it can be understood that the first two-dimensional display sub-area is used for displaying the first fluorescence two-dimensional image and the first white light two-dimensional image corresponding to the first sub three-dimensional image, and the second two-dimensional display sub-area is used for displaying the third fluorescence two-dimensional image and the third white light two-dimensional image corresponding to the third sub three-dimensional image, so that the block division of the white light and the fluorescence display area is carried out.

And displaying the first fluorescence two-dimensional image and the first white light two-dimensional image in the white light display area and the fluorescence display area of the first two-dimensional display sub-area, and sequentially displaying the third fluorescence two-dimensional image and the third white light two-dimensional image in the first fluorescence sequence in the white light display area and the fluorescence display area of the second two-dimensional display sub-area.

According to the technical scheme provided by the invention, the first fluorescence two-dimensional image and the first white light two-dimensional image are displayed in the white light display area and the fluorescence display area of the first two-dimensional display sub-area, and the third fluorescence two-dimensional image and the third white light two-dimensional image are sequentially displayed in the white light display area and the fluorescence display area of the second two-dimensional display sub-area according to the sequence in the first fluorescence sequence, wherein the first fluorescence sequence is that the third fluorescence two-dimensional images are sequenced from large to small according to the first total number, and the higher the first total number is, the higher the corresponding relevance is.

In a possible embodiment, the technical solution provided by the present invention further includes:

According to the technical scheme provided by the invention, if the number of the third fluorescent two-dimensional images is more than one, the third fluorescent two-dimensional images are sequenced from large to small according to the corresponding first total number to generate a first fluorescent sequence, and it can be understood that the larger the first total number is, the higher the corresponding degree of correlation is, the more the arrangement is, the priority display is convenient to carry out, so that a doctor can preferentially check the most relevant image area and can better fit the actual diagnosis process.

and selecting fluorescent pixel points adjacent to the non-fluorescent pixel points in the first fluorescent two-dimensional image to obtain a first contour fluorescent pixel point set.

According to the technical scheme provided by the invention, the system can select the fluorescent pixel points adjacent to the non-fluorescent pixel points in the first fluorescent two-dimensional image, and it can be understood that the fluorescent pixel points adjacent to the non-fluorescent pixel points are the edge of the pathological change region, namely the boundary part of the pathological change region and the healthy cells, so that a first contour fluorescent pixel point set can be obtained, and the subsequent system can conveniently splice the pathological change region according to the picture position to generate a complete pathological change part.

And selecting fluorescent pixel points adjacent to the non-fluorescent pixel points in the third fluorescent two-dimensional image to obtain at least one second contour fluorescent pixel point set.

According to the technical scheme provided by the invention, the system can select the fluorescent pixel points adjacent to the non-fluorescent pixel points in the third fluorescent two-dimensional image, and can understand that the third fluorescent two-dimensional image is a region associated with a part of the region selected by a doctor, and the boundary part between the third fluorescent two-dimensional image and the healthy cells can be obtained by taking the fluorescent pixel points adjacent to the non-fluorescent pixel points as the edge of the lesion region, so that at least one second contour fluorescent pixel point set is obtained, the corresponding contour sets can be spliced conveniently according to the respective positions of the images to obtain a complete lesion part, and the system can automatically splice the lesion region to form a complete target lesion region for the doctor to observe conveniently.

And the first contour fluorescence pixel point set and the second contour fluorescence pixel point set are spliced based on the positions of the first fluorescence two-dimensional image and the third fluorescence two-dimensional image to obtain a target contour fluorescence two-dimensional image.

According to the technical scheme provided by the invention, the first contour fluorescence pixel point set and the second contour fluorescence pixel point set are spliced based on the positions of the first fluorescence two-dimensional image and the third fluorescence two-dimensional image to obtain a complete target contour fluorescence two-dimensional image, and it can be understood that after a doctor selects a target region, a system not only can automatically push the associated region, but also can splice the associated region into a complete lesion region according to the contour according to the position of the associated region, for example: the region that the doctor selected is the upside image, the image of automatic propelling movement is the downside image of this image, the first profile fluorescence pixel point set that corresponds is in the upside, second profile fluorescence pixel point set is in the downside, both splice and form a complete pathological change region, make things convenient for the doctor to diagnose, and splice into complete regional back, can only show pathological change part, can reduce and show healthy regional, make pathological change part detail show more abundant, comparatively make things convenient for the doctor to diagnose.

According to the technical scheme provided by the invention, the third two-dimensional display sub-region is established again in the two-dimensional image display region and is used for displaying the fluorescence two-dimensional image of the target contour, and the three-dimensional image of the whole lesion region, the first fluorescence two-dimensional image selected by a doctor, the first white light two-dimensional image, the automatically-pushed third fluorescence two-dimensional image and the third white light two-dimensional image can be displayed in the final image display frame, and the first fluorescence two-dimensional image and the third fluorescence two-dimensional image are spliced into the fluorescence two-dimensional image of the whole lesion region, so that all details of the whole lesion region and part of the lesion region selected by the doctor are displayed to the doctor, and the doctor can conveniently perform diagnosis.

In order to better implement the fluorescence three-dimensional imaging method based on data processing provided by the present invention, the present invention further provides a fluorescence three-dimensional imaging system based on data processing, as shown in fig. 3, including:

the calling module is used for displaying the fluorescent three-dimensional fusion image, and calling a first fluorescent two-dimensional image and a first white light two-dimensional image corresponding to a first sub three-dimensional image for displaying if the first sub three-dimensional image is judged to be selected by a user;

As shown in fig. 4, which is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention, theelectronic device 50 includes: aprocessor 51, amemory 52 and a computer program; wherein

Amemory 52 for storing the computer program, which may also be a flash memory (flash). The computer program is, for example, an application program, a functional module, or the like that implements the above-described method.

Aprocessor 51 for executing the computer program stored in the memory to implement the steps performed by the apparatus in the above method. Reference may be made in particular to the description relating to the preceding method embodiment.

Alternatively, thememory 52 may be separate or integrated with theprocessor 51.

When thememory 52 is a device independent of theprocessor 51, the apparatus may further include:

abus 53 for connecting thememory 52 and theprocessor 51.

The present invention also provides a readable storage medium, in which a computer program is stored, which, when being executed by a processor, is adapted to implement the methods provided by the various embodiments described above.

The readable storage medium may be a computer storage medium or a communication medium. Communication media includes any medium that facilitates transfer of a computer program from one place to another. Computer storage media may be any available media that can be accessed by a general purpose or special purpose computer. For example, a readable storage medium is coupled to a processor such that the processor can read information from, and write information to, the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may reside in an Application Specific Integrated Circuits (ASIC). Additionally, the ASIC may reside in user equipment. Of course, the processor and the readable storage medium may also reside as discrete components in a communication device. The readable storage medium may be a read-only memory (ROM), a random-access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

The present invention also provides a program product comprising execution instructions stored in a readable storage medium. The at least one processor of the device may read the execution instructions from the readable storage medium, and the execution of the execution instructions by the at least one processor causes the device to implement the methods provided by the various embodiments described above.

In the above embodiments of the apparatus, it should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of hardware and software modules.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and these modifications or substitutions do not depart from the spirit of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A fluorescence three-dimensional imaging method based on data processing is characterized by comprising the following steps:

based on the two-dimensional image extraction device, the two-dimensional image extraction device extracts images of pathological change tissues under fluorescence and white light to obtain fluorescence two-dimensional images and white light two-dimensional images of different region positions of the pathological change tissues, and the method comprises the following steps:

controlling incident light to perform annular scanning under the same incident angle and different direction angles of 360 degrees based on a two-dimensional image extraction device, and respectively performing image extraction on lesion tissues under fluorescence and white light to obtain fluorescence two-dimensional images and white light two-dimensional images at different region positions of the lesion tissues;

determining a plurality of second sub three-dimensional images according to the region positions of the first sub three-dimensional images, and determining at least one third fluorescent two-dimensional image according to the number of fluorescent pixel points in the second fluorescent two-dimensional image corresponding to each second sub three-dimensional image, wherein the fluorescent pixel points are pixel points in a preset fluorescent pixel value interval, and the second sub three-dimensional images are sub three-dimensional images adjacent to the first sub three-dimensional images;

generating an image display frame according to the number of the third fluorescent two-dimensional images, and displaying the fluorescent three-dimensional fusion image, the first fluorescent two-dimensional image, the first white light two-dimensional image, the third fluorescent two-dimensional image and a third white light two-dimensional image corresponding to the third fluorescent two-dimensional image on the basis of the image display frame;

the determining a plurality of second sub three-dimensional images according to the region position of the first sub three-dimensional image, and determining at least one third fluorescent two-dimensional image according to the number of fluorescent pixel points in the second fluorescent two-dimensional image corresponding to each second sub three-dimensional image includes:

and if the first total number is larger than the preset number, taking the second fluorescence two-dimensional image as a third fluorescence two-dimensional image.

2. The method of claim 1,

the method comprises the steps of extracting images of lesion tissues under fluorescence and white light based on a two-dimensional image extraction device to obtain fluorescence two-dimensional images and white light two-dimensional images of different region positions of the lesion tissues, and generating a fluorescence three-dimensional fusion image corresponding to the lesion tissues according to the fluorescence two-dimensional images and the white light two-dimensional images, and comprises the following steps:

3. The method of claim 2,

the method for dividing the fluorescence three-dimensional fusion image into a plurality of sub three-dimensional images according to the region positions to establish a preset storage space corresponding to each sub three-dimensional image and storing the fluorescence two-dimensional image and the white light two-dimensional image of the corresponding region positions in the preset storage space comprises the following steps:

4. The method of claim 3,

the obtaining of the corresponding actual division size according to the actual size of the fluorescence three-dimensional fusion image comprises:

the actual division size is obtained by the following formula,

wherein,

in order to actually divide the size of the wafer,

in order to preset the division size,

is the actual size of the fluorescence three-dimensional fusion image,

is a preset size of the fluorescence three-dimensional fusion image,

is an attribute value of the actual division size.

5. The method of claim 4,

the counting of the number of the fluorescence pixel points which are directly connected with the boundary fluorescence pixel points and indirectly connected with the boundary fluorescence pixel points in the second fluorescence two-dimensional image to obtain a first total number comprises the following steps:

6. The method of claim 5,

the generating of the image display frame according to the number of the third fluorescent two-dimensional images, and displaying the fluorescent three-dimensional fusion image, the first fluorescent two-dimensional image, the first white light two-dimensional image, the third fluorescent two-dimensional image, and the third white light two-dimensional image corresponding to the third fluorescent two-dimensional image based on the image display frame include:

7. The method of claim 6, further comprising:

8. The method of claim 7, further comprising:

selecting fluorescent pixel points adjacent to non-fluorescent pixel points in the first fluorescent two-dimensional image to obtain a first contour fluorescent pixel point set;

9. A fluorescence three-dimensional imaging device based on data processing is characterized by comprising:

the fusion module is used for extracting images of the lesion tissue under fluorescence and white light based on the two-dimensional image extraction device to obtain fluorescence two-dimensional images and white light two-dimensional images at different area positions of the lesion tissue, and generating a fluorescence three-dimensional fusion image corresponding to the lesion tissue according to the fluorescence two-dimensional images and the white light two-dimensional images;

based on two-dimensional image extraction device carries out image extraction under fluorescence and white light to pathological change tissue, obtains the fluorescence two-dimensional image and the white light two-dimensional image of pathological change tissue different region position department, includes:

the determining module is used for determining a plurality of second sub three-dimensional images according to the region positions of the first sub three-dimensional images, and determining at least one third fluorescent two-dimensional image according to the number of fluorescent pixel points in the second fluorescent two-dimensional image corresponding to each second sub three-dimensional image, wherein the fluorescent pixel points are pixel points in a preset fluorescent pixel value interval, and the second sub three-dimensional images are sub three-dimensional images adjacent to the first sub three-dimensional images;

the display module is used for generating an image display frame according to the number of the third fluorescent two-dimensional images and displaying the fluorescent three-dimensional fusion image, the first fluorescent two-dimensional image, the first white light two-dimensional image, the third fluorescent two-dimensional image and the third white light two-dimensional image corresponding to the third fluorescent two-dimensional image on the basis of the image display frame;