Detailed Description
The technical solutions of the present invention will be clearly and completely described in connection with the embodiments, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The cell interpretation method at present is to interpret corresponding cells through a machine learning algorithm, and the reliability of the interpretation result obtained by the machine learning algorithm is lower. Based on the above, the cell interpretation method and system provided by the embodiment of the invention uses a plurality of artificial intelligence algorithms to interpret cells, and determines the final interpretation result based on the comparison results of a plurality of interpretation results, so that the interpretation flow is more strict, the possibility of misinterpretation and omission of interpretation is reduced, and the accuracy and reliability of cell interpretation are further improved.
For the convenience of understanding the present embodiment, a cell interpretation method disclosed in the embodiment of the present invention will be described in detail.
Embodiment one:
Referring to fig. 1, an embodiment of the present invention provides a cell interpretation method, which may include the steps of:
step S101, obtaining cell data corresponding to the cell image, and respectively judging the cells by utilizing a plurality of AI algorithms based on the cell data to obtain a plurality of judging results of the cells.
In the embodiment of the invention, one cell image contains a plurality of cells, so that the cell data corresponding to the cell image is also the data of a plurality of cells. AI algorithms (ARTIFICIAL INTELLIGENCE algorism) include, but are not limited to, deep learning algorithms (e.g., various image segmentation and recognition algorithms based on convolutional neural networks) and machine learning algorithms (e.g., random forests, support vector machines, neural networks, etc.). And (3) selecting a plurality of different AI algorithms at will, wherein the processes of the AI algorithms are completely independent when the cells are interpreted. Wherein the interpretation result depends on the interpretation content, for example: the interpretation results can be divided into two main categories: cell marker recognition results, which may refer to the type of intracellular fluorescent spots, the number of each type of intracellular fluorescent spots, and cell overall interpretation results, which may include: cell total number, cell classification results, which in turn may include: cell type, and number of cells per cell type. When the cell classification result is taken as an interpretation result (i.e., a plurality of interpretation results of the above-described cells), the cell type and the number of cells under each type can be determined. For example, if the AI algorithms are two, the first AI algorithm corresponds to a first interpretation result and the second AI algorithm corresponds to a second interpretation result. The first interpretation results are that the number of the cell types is 3, namely, type A, type B and type C, the number of the cells corresponding to the type A is 10, the number of the cells corresponding to the type B is 6, and the number of the cells corresponding to the type C is 20. The second interpretation result is that the number of the cell types is 2, namely the type A and the type B, and the number of the cells corresponding to the type A is 10, and the number of the cells corresponding to the type B is 6. Therefore, the same cell data is interpreted by using two different AI algorithms, so that misjudgment and missed judgment can be reduced.
It should be noted that an AI algorithm is a combination of one or more deep learning algorithms or other non-artificial intelligence algorithms, and each AI algorithm can perform three steps of cell segmentation recognition, cell marker recognition and cell classification. Through cell segmentation operation, the cell structure after segmentation can be identified clearly, and other structures without cells can be effectively eliminated through cell identification. Through cell marker recognition, markers required for cell interpretation can be obtained. By interpretation, cell classification results can be obtained.
The embodiment of the invention can respectively interpret cells by adopting two deep learning network algorithms, namely Mask R-CNN and Yolo. It should be noted that the two deep learning network algorithms, mask R-CNN and Yolo, are consistent in interpretation process when interpreting cells in the same cell image. The following description is made by taking Mask R-CNN as an example: step 21, first, cell characteristics are identified for the cells, including but not limited to: cell contour, cell staining signal, cell tissue; step 22, then carrying out intracellular marker recognition on the cells after cell characteristic recognition; step 23: determining the interpretation result of Mask R-CNN on cells based on the cell marker recognition result obtained in the step 22; the identification result of the cell marker can realize classification of different types of fluorescent spots in cells, and the interpretation result of Mask R-CNN on cells can be called as a cell classification result. For example: mask R-CNN identified 10 cells, 2 red spots and 3 green spots in 2 cells; wherein, the fluorescent points with different colors are used for representing the fluorescent points with different types. In addition, 5 red fluorescent spots and 6 green fluorescent spots are arranged in 8 cells, and preset interpretation standards are as follows: the cell type containing 2 red spots and 3 green spots is type one, and the cell type containing 5 red spots and 6 green spots is type two, so that the result of the interpretation is that there are two cell types, type one and type two, respectively, and the number of cells corresponding to type one is 2, and the number of cells corresponding to type two is 8.
In the implementation of the step S101 of the present application, in order to achieve the high efficiency of interpretation, the interpretation process is continuously performed, that is, two AI algorithms continuously interpret the cell data corresponding to each cell image, and in order to ensure that the two AI algorithms interpret the cells in the same cell image, that is, in order to ensure the consistency of interpretation, the cell data in the same path and the same position need to be acquired.
And S102, comparing a plurality of interpretation results of the cells so as to enable a plurality of AI algorithms to mutually monitor and obtain a comparison result. The automatic supervision of multiple AI algorithms can be realized through the comparison of multiple interpretation results, the algorithm with poor effect can be timely found, and then timely trained, so that the reliability of the algorithm is ensured, an effect for supervising and promoting the accuracy of the algorithm can be indirectly formed, and the training is continuously carried out, so that the accuracy is improved.
Step S103, determining the final interpretation result of the cells based on the comparison result.
In the embodiment of the invention, the comparison results include, but are not limited to: the multiple interpretation results of the cells are consistent, part of the interpretation results of the cells are consistent, the other part of the interpretation results are inconsistent, and the multiple interpretation results of the cells are inconsistent. The final interpretation result may refer to an interpretation result analysis report of the cells. The interpretation result analysis report includes, but is not limited to, cell data, a plurality of interpretation results of cells, and a manual review result. The step S103 may determine the final determination result in the following two ways: the first mode is to completely adopt a manual rechecking mode, namely, judging by judging through professional experience of professionals; mode two: the tool is assisted by combining with a manual review mode, namely, the tool is judged to be automatically compared by utilizing the characteristic of targeted research and development, and the characteristics include but are not limited to: cell morphology, cell staining signals, signal distribution, and then manually rechecked.
According to the cell interpretation method provided by the embodiment of the invention, cell data corresponding to a cell image are firstly obtained, and cells are respectively interpreted by utilizing a plurality of AI algorithms based on the cell data, so that a plurality of interpretation results of the cells are obtained; then comparing a plurality of interpretation results of the cells; and finally determining the final interpretation result of the cells based on the comparison result. On one hand, the embodiment of the invention can discover the algorithm with poor effect in time through mutual supervision of multiple algorithms, train in time, further ensure the reliability of the algorithm, indirectly form an effect of supervising and promoting the accuracy of the algorithm, train continuously, improve the accuracy, finally ensure the reliability of cell judgment, and improve the accuracy through training. On the other hand, the embodiment of the invention utilizes various artificial intelligence algorithms to interpret cells, and determines the final interpretation result based on the comparison results of various interpretation results, so that the interpretation flow is more strict, the possibility of misinterpretation and missed interpretation is reduced, and the accuracy and reliability of cell interpretation are further improved.
Further, referring to fig. 2, step S103 may include the steps of:
Step S201, if the comparison result is that the plurality of interpretation results of the cells are consistent, the cell data and the target interpretation result of the cells are sent to the first client, wherein the target interpretation result is one of the plurality of interpretation results.
In the embodiment of the invention, two AI algorithms are adopted to interpret cells, and the first interpretation result of the cells is identical with the second interpretation result of the cells, so that the first interpretation result sent to the first client has the same effect as the second interpretation result sent to the first client. Therefore, only the first interpretation result is sent to the first client, and the first interpretation result is the target interpretation result.
Step S202, receiving a manual review result sent by the first client based on the target interpretation result, wherein the manual review result is used for representing the correctness of the interpretation result.
Step S203, if the plurality of interpretation results are determined to be correct based on the manual review result, generating an interpretation result analysis report of the cells.
Further, referring to fig. 3, after step S203, the method further includes:
Step S204, the interpretation result analysis report is sent to the second client side, so that the user confirms the interpretation result analysis report.
Step S205, receiving a confirmation result of the second client side on the interpretation result analysis report.
In step S206, if the result is confirmed to be interpretation non-objection, the cells and the interpretation result analysis report of the cells are stored in the user non-objection database.
In the embodiment of the invention, the user receives the interpretation result analysis report of the cell through the second client, can check the interpretation result analysis report of the cell after receiving, and can confirm the interpretation result analysis report to obtain a confirmation result, wherein the confirmation result comprises two types of interpretation dissimilarity and interpretation dissimilarity.
Further, referring to fig. 3, after step S205, the method further includes:
Step S207, if the result of the confirmation is that the interpretation is disagreement, the cells are marked, and the marked cells and the result of the confirmation of the cells are sent to the third client side, so that the staff can manually review the cells to obtain a manual review confirmation result.
Step S208, receiving a manual review confirmation result sent by the third client.
In the embodiment of the invention, the manual review confirmation result comprises: determining that interpretation is not inconsistent and determining that interpretation is inconsistent. And if the manual review confirmation result is that the interpretation is not disagreement, adding the manual review confirmation result into the interpretation result analysis report to update the interpretation result analysis report.
Step S209, if the result of the manual review is that the interpretation is determined to be objection, storing the marked cells and a plurality of interpretation results of the cells into a user objection database.
In the embodiment of the invention, the user can correct the interpretation result of the cell objection in the objection database of the user. When the number of cells in the user objection database reaches a certain number, the AI algorithm is relearned and retrained, and the data training set utilized in training can select the cells in the user objection database. Or the cells in the user objection database are used for relearning and retraining the AI algorithms at regular intervals, so that the AI algorithms are optimized, and the cell interpretation system is optimized.
Further, referring to fig. 4, the following steps may be further included after step S102:
step S301, if the comparison result is that the plurality of interpretation results of the cells are inconsistent, the cell data and the plurality of interpretation results of the cells are sent to the first client to obtain a manual review result.
Step S302, receiving a manual review result fed back by the first client, and sending the manual review result to the second client as an interpretation result analysis report of the cells, so that a user confirms the interpretation result analysis report.
In the embodiment of the invention, taking the case of interpreting cells by adopting two AI algorithms, when the first interpretation result of the cells is inconsistent with the second interpretation result of the cells, the result of manual rechecking is that the interpretation is inconsistent. The inconsistent interpretation errors can include three cases, case 1, where the first interpretation of the cells is correct and the second interpretation of the cells is incorrect; case 2, the first interpretation of the cells is incorrect and the second interpretation of the cells is correct; case 3, both the first interpretation of the cells and the second interpretation of the cells are erroneous, for example: the first interpretation of cells is 5 cells of type one, 2 cells of type three, 3 cells of type one, 4 cells of type three, and the actual result of cells is 2 cells of type one, 6 cells of type three. And the user confirms the analysis report of the interpretation results, namely, whether the two interpretation results are correct or not.
Further, referring to fig. 4, after step S102, the method further includes:
Step S303, if the comparison result is that the interpretation results of the cells are inconsistent, storing the cell data and the interpretation results of the cells into an interpretation inconsistent database.
And S304, counting and judging the number of the cells in the inconsistent database, and retraining a plurality of AI algorithms when the number of the cells in the inconsistent database reaches the preset number.
By counting and judging the number of cells in the inconsistent database, whether the correctness of the AI algorithm reaches a preset threshold value can be determined. The fewer the number of cells in the interpretation inconsistent database, the more accurate the predictions for the various AI algorithms are. For example: when two AI algorithms are utilized for interpretation, as the accuracy of the first AI algorithm reaches a preset threshold, the interpretation effect is good, and the accuracy of the second AI algorithm does not reach the preset threshold, and the interpretation effect is poor, the number of cells in the inconsistent interpretation database can reach the preset number in a short time. Therefore, the embodiment can prevent each AI algorithm from having lower accuracy in application by mutually supervising a plurality of AI algorithms, thereby achieving the effects of algorithm reinforcement training and further improvement.
Whether the first AI algorithm is model intensive training or the second AI algorithm is model optimized training, the data training set and the data testing set are needed to be used on the model which is trained previously. The data training set may contain new training data in addition to historical training data. The new training data can be generated after the comparison of two interpretation results of the cells, and specifically, the cell data corresponding to the cells in the three libraries of the interpretation inconsistent database, the interpretation error database and the user objection database can be marked according to preset training requirements to form new training data qualified in specification.
In the embodiment of the invention, two AI algorithms are adopted to interpret cells, when the number of the cells in the interpretation inconsistent database reaches the preset number, the two AI algorithms are relearned and retrained, and the data training set utilized in training can select the cells in the interpretation inconsistent database. The retraining method and the retraining device aim to tune the two AI algorithms until the accuracy of the two AI algorithms reaches a preset threshold. Specifically, cells in the interpretation inconsistent database can be placed into the data training set and the data testing set according to a certain proportion. And then respectively training a first AI algorithm and a second AI algorithm through the cell data in the data training set, and respectively adjusting the first AI algorithm and the second AI algorithm according to the test result of the data test set until the test results of the first AI algorithm and the second AI algorithm on the data test set are consistent.
The condition of retraining the multiple AI algorithms is not limited to the condition that the number of cells in the inconsistent interpretation database reaches the preset number, so that the condition that the number of cells in the incorrect interpretation database reaches the preset number or the condition that the number of cells in the inconsistent interpretation database reaches the preset number can be used as the condition of retraining the multiple AI algorithms.
Specifically, the number of cells in the error database is counted, and when the number of cells in the error database reaches the preset number, the AI algorithm is retrained.
Counting the number of cells in the interpretation objection database, and retraining a plurality of AI algorithms when the number of cells in the interpretation objection database reaches a preset number.
Further, after step S202, the method further includes:
If the result of the manual rechecking is interpretation error, storing the cell and a plurality of interpretation results of the cell into an interpretation error database.
In the embodiment of the invention, two AI algorithms are taken as an example for interpreting cells, and if the first interpretation result and the second interpretation result of the cells are consistent, but the two interpretation results are wrong, the result of the manual rechecking is wrong interpretation. For example: the first interpretation of the cells was 3 red spots in the cell, the second interpretation of the cells was 3 red spots in the cell, and the cells were actually 2 red spots. The cells with the interpretation errors and the first interpretation results of the cells are stored in an interpretation error database. The embodiment of the invention can train two AI algorithms by taking cells in the interpretation error database as a training set.
In the embodiment of the invention, taking two AI algorithms for interpreting cells as an example, FIG. 5 provides a flow chart of another cell interpretation method, and the two AI algorithms are continuously trained and optimized, so that the reliability of the algorithm is ensured, the accuracy is improved, the accuracy of the interpretation result can be ensured, the operation of manual rechecking and the operation of user confirmation exist, the interpretation process is ensured to be strict, the possibility of misinterpretation and missed interpretation is reduced, and the reliability of the cell interpretation result is improved.
Embodiment two:
referring to fig. 6, an embodiment of the present invention provides a cell interpretation system, which may include the following modules:
the acquiring and interpreting module 11 is configured to acquire cell data corresponding to a cell image, and respectively interpret cells by using a plurality of AI algorithms based on the cell data to obtain a plurality of interpretation results of the cells;
The comparison module 12 is used for comparing a plurality of interpretation results of the cells so as to realize mutual automatic supervision of a plurality of AI algorithms;
A determining module 13, configured to determine a final interpretation result of the cell based on the comparison result;
further, referring to fig. 7, the determination module 13 includes the following units:
A sending unit 131, configured to send a target interpretation result of the cell and the cell to the first client if the comparison result is that the plurality of interpretation results of the cell are consistent, where the target interpretation result is a result of the plurality of interpretation results;
A receiving unit 132, configured to receive a manual review result sent by the first client based on the target interpretation result, where the manual review result is used to characterize correctness of the interpretation result;
And a generating unit 133 configured to generate an interpretation result analysis report of the cells if it is determined that the plurality of interpretation results are correctly interpreted based on the manual review result.
Further, referring to fig. 8, the system further includes the following modules:
the first sending module 14 is configured to send the interpretation result analysis report to the second client, so that the user confirms the interpretation result analysis report.
The first receiving module 15 is configured to receive a confirmation result of the second client to the interpretation result analysis report.
The first storage module 16 is configured to store the cell and the interpretation result analysis report of the cell in the user non-objection database if the result is confirmed to be interpretation non-objection.
Further, referring to fig. 8, the system further includes:
The marking module 17 is configured to mark the cells if the confirmation result is that the interpretation is inconsistent, and send the marked cells and the confirmation result of the cells to the third client, so that the worker performs manual review on the cells to obtain a manual review confirmation result;
A second receiving module 18, configured to receive a manual review confirmation result sent by the third client;
and a second storage module 19, configured to store the marked cells and the multiple interpretation results of the cells in the user objection database if the result of the manual review confirmation is that the interpretation has objection.
Further, referring to fig. 8, the system may further include the following modules:
the second sending module 20 is configured to send the cell data and the multiple interpretation results of the cells to the first client if the comparison results are inconsistent, so as to obtain a manual review result;
and the third receiving module 21 is configured to receive the manual review result fed back by the first client, and send the manual review result as an interpretation result analysis report of the cell to the second client, so that the user confirms the interpretation result analysis report.
Further, referring to fig. 8, the system further includes: the third storage module 22 is configured to store the cell data and the multiple interpretation results of the cells into an interpretation inconsistency database if the comparison result is that the multiple interpretation results of the cells are inconsistent;
the statistics module 23 is used for counting the number of cells in the interpretation inconsistent database, and retraining a plurality of AI algorithms when the number of cells in the interpretation inconsistent database reaches a preset number;
further, referring to fig. 8, the system further includes: and the fourth storage module 24 stores the cells and the multiple interpretation results of the cells into the interpretation error database if the result of the manual review is interpretation error.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Furthermore, the terms "first," "second," "third," "fourth," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.