Disclosure of Invention
The present invention has been made in view of the above-mentioned problems with existing patient condition tracking methods and systems.
Therefore, the problem to be solved by the invention is that the traditional image registration method generally depends on global rigid transformation, although the effect is better on global image alignment, the method cannot cope with local morphological changes of a focus area, particularly the focus tracking with larger time changes of a special focus and the like, shows larger limitation, and secondly, although a deep convolutional neural network is widely applied to focus segmentation, the problem of fuzzy boundary and high false-division rate still can be faced when the segmentation is carried out by simply relying on CNN, particularly the segmentation precision of the focus area is difficult to ensure under different imaging conditions or noise interference, and further the dynamic change trend of the focus is difficult to be accurately reflected, particularly in long-term tracking, the volume increase rate and the change trend of the focus are difficult to be effectively predicted.
In order to solve the technical problems, the invention provides the following technical scheme that the patient condition tracking method comprises the following steps:
Collecting focus image data of a patient, and carrying out rigid registration on the focus image data through affine transformation;
Predicting focus probability distribution based on a depth convolution neural network, and performing focus segmentation and marking based on a Conditional Random Field (CRF);
Performing non-rigid registration on focus region image data by using the B-spline basis function description local displacement to generate a final image;
and selecting a focus area for volume calculation, and determining the focus volume increase rate and change rate by using a nonlinear differential equation.
As a preferable mode of the patient condition tracking method, the method for acquiring the patient focus image data comprises the following steps:
image data acquisition is carried out according to specific illness condition requirements of patients;
determining an image data acquisition frequency based on doctor experience;
preprocessing is performed based on the image data, including denoising and gray scale normalization of the image data, and image cropping processing.
As a preferred embodiment of the patient condition tracking method of the present invention, the rigid registration of lesion image data by affine transformation includes:
based on the preprocessed image data, performing preliminary alignment on the image data of multiple time points through affine transformation, defining an affine transformation matrix, and calculating a new position of each pixel through matrix multiplication and vector addition operation to perform rigid registration, wherein the rigid registration is expressed as follows: Wherein the method comprises the steps ofRepresenting the post-change coordinates of the pixel coordinate points,Representing the pixel coordinates of the original image,B is affine transformation matrix and displacement vector;
Determining a reference image based on a training set, selecting a mean square error MSE as a similarity measurement function, performing iterative optimization by an Adam optimizer and by a gradient descent method aiming at minimizing the difference between the target image and the reference image, stopping iteration when the loss calculated in the continuous iteration process is not obviously reduced, determining the optimal A and b, and outputting an optimal affine transformation matrix;
Transforming the coordinates of each pixel point to new position coordinates by rigid registrationAn alignment image is generated.
As a preferable scheme of the patient condition tracking method, the method for predicting the lesion probability distribution based on the depth convolution neural network, and performing lesion segmentation and marking based on a conditional random field CRF comprises the following steps:
Constructing a focus area model based on a depth convolutional neural network, wherein the focus area model comprises an input layer, a convolutional layer, a maximum pooling layer, a full-connection layer and an output layer;
using a training set training model, selecting a cross entropy loss function to calculate the difference between a focus area model prediction result and an actual label, using an Adam optimizer to perform gradient descent optimization, updating parameters of the model, and stopping iterating the output model when the loss of the model is not obviously reduced in a continuous iterating process;
Obtaining a focus probability distribution map based on the alignment image based on a focus region segmentation prediction result output by the focus region model;
based on the conditional random field CRF, the probability that each pixel belongs to the lesion area is taken as a single-point term, the similarity between the pixel points is defined as a double-point term, and an energy function is defined based on the single-point term and the double-point term, and is expressed as:;; WhereinIndicating the cost to which the ith pixel belongs to the lesion,Representing the probability that the ith pixel belongs to the lesion area,Representing the similarity cost of adjacent i-th and j-th pixels,AndRepresenting the spatial coordinates of the i-th pixel and the j-th pixel respectively,The weight coefficient is represented by a number of weight coefficients,AndRespectively representing the gray values of the ith pixel and the jth pixel,AndRespectively representing a spatial scale parameter and a color scale parameter,Represents the total energy function value, N represents the total number of pixel points,Representing the adjacent pixel domain of the ith pixel;
Calculating loss between lesion prediction result and actual label of conditional random field CRF by using training set and selecting cross entropy loss function, gradient descent optimizing by using Adam optimizer, updating conditional random field CRF parameters including、AndStopping iteration when the calculation loss is not obviously reduced in the continuous iteration process, outputting model parameters, and updating the model;
Optimizing an energy function by using a Mean-Field Approximation Mean field approximation method, and determining an optimization formula, wherein the optimization formula is expressed as follows: WhereinThe normalization factor is represented as such,The probability of classification is represented by a number of classes,Representing a classification probability of neighboring pixels;
taking a prediction probability map of a focus area model as an initial valueDetermining;
Based onUpdating the classification probability of each pixel as in successive iterationsStopping iteration when the variation of the model is no longer obviously reduced, and outputting the final model;
Based on the final resultGenerating a corresponding focus area probability map, dividing pixel values of the probability map based on the historical average value as a dividing threshold value, and marking the focus area.
As a preferable scheme of the patient condition tracking method, the method for performing non-rigid registration on focus area image data by using the B-spline basis function to describe local displacement, and generating a final image comprises the following steps:
Generating grids based on focus areas of the image data, defining grid nodes as control points, and performing non-rigid registration;
the local displacement is described using a B-spline basis function based on each control point, expressed as:; WhereinRepresents the displacement amount of the pixel x, n represents the number of control points adjacent to the pixel,The displacement vector representing the control point i,The B-spline basis function is represented,Representing the coordinates of the i-th control point,Representing a third-order B-spline basis function;
Determining a reference image based on the training set, selecting a mean square error MSE as a similarity measure function, the object being to minimize the difference between the target image and the reference image, expressed as: WhereinRepresenting the non-rigid registration calculation error,Representing the total number of pixels,Representing the coordinates of the i-th pixel point,For the corresponding pixel point coordinates in the reference image, M represents the total number of control points,Represents a smoothness regularization parameter, which is determined based on historical data,Representing the gradient of the displacement vector of the control point j,Representing pixel pointsDisplacement of (2);
Displacement according to current control point using Levenberg-Marquardt optimization algorithmCalculating the value of the non-rigid registration calculation error, and calculating the displacement of the objective function for each control pointThen updating the control point displacement, stopping iteration when the calculated error in the continuous optimization process is not obviously reduced any more, and optimizing the displacement of the control point;
Calculating pixel point displacement based on control point displacement by non-rigid registrationAnd further adjusting the positions of the pixel points based on the aligned images to generate a final image.
As a preferable mode of the patient condition tracking method of the invention, the selecting focus area to perform volume calculation comprises,
Based on the image scanning device, three-dimensional image data are formed through stacking of multiple layers of image data, and based on the final image generated through non-rigid registration, a focus change area is determined and voxels of the focus change area are selected for volume calculation.
As a preferred embodiment of the patient condition tracking method of the present invention, the determining the lesion volume increase rate and change rate using a nonlinear differential equation comprises:
fitting the volume change data of the lesion using a nonlinear differential equation is expressed as: WhereinThe lesion volume at time t is indicated,Indicating the rate of increase of the lesion volume,A maximum limit value representing a lesion volume;
Calculating a minimized difference between the observed volume and the predicted volume using the historical data and using the squared error as a loss function;
Calculating loss by using nonlinear least square method, stopping iteration when the loss of the model is not obviously reduced in the continuous iteration process, and outputting final K and K;
The rate of change of the growth rate is determined based on the change of the lesion volume growth rate at different times, and the rate of change is evaluated according to the experience of the physician.
It is another object of the present invention to provide a system for a patient condition tracking method comprising:
The image data processing module is used for collecting focus image data of a patient and carrying out image preprocessing;
a rigid registration module for rigid registration by affine transformation;
the focus recognition and segmentation module is used for predicting probability distribution of focus areas based on the depth convolution neural network and carrying out focus segmentation and marking by using a conditional random field;
The non-rigid registration module is used for describing local displacement through the B-spline basis function and carrying out non-rigid registration;
The focus volume calculation module is used for selecting a focus area to perform volume calculation based on the image data after non-rigid registration;
a volume change analysis module for determining a volume increase rate and a change rate of the lesion using a nonlinear differential equation.
A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the patient condition tracking method when executing the computer program.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the patient condition tracking method described above.
The method has the advantages that the accuracy of focal region segmentation is improved through the combination of the deep convolutional neural network and the conditional random field, human intervention is reduced, global alignment of images can be quickly realized through the combination of rigid registration and model identification, the change of a focal region can be carefully tracked, the displacement of pixels can be smoothly interpolated between control points through the non-rigid registration of a B-spline basis function, the generated images can be accurately aligned with the form and the position of the focal region, reliable basic data are provided for follow-up focus change analysis, form change quantification and the like, fine and reliable focus volume measurement can be provided through voxel calculation, particularly for the condition of complex focus form or relatively fuzzy focus edges, the model parameters can be fitted through a nonlinear least square method by utilizing the historical focus volume data, the volume change of a focus in future time can be accurately predicted, and doctors can be helped to predict the development trend of the focus.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways other than those described herein, and persons skilled in the art will readily appreciate that the present invention is not limited to the specific embodiments disclosed below.
Further, reference herein to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic can be included in at least one implementation of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.
Embodiment 1, referring to fig. 1, is a first embodiment of the present invention, which provides a patient condition tracking method, including:
s1, collecting focus image data of a patient, and carrying out rigid registration on the focus image data through affine transformation;
preferably, patient lesion image data is acquired, including,
Image data acquisition is carried out according to specific illness requirements of a patient, and image data acquisition can be carried out by adopting CT, MRI, PET-CT or ultrasonic imaging and other imaging examinations, wherein CT is used for hard tissue imaging, MRI is used for soft tissue imaging, PET-CT is suitable for metabolic imaging, and ultrasonic is suitable for real-time dynamic imaging;
determining an image data acquisition frequency based on doctor experience;
preprocessing is performed based on the image data, including denoising and gray scale normalization of the image data, and image cropping processing.
Through determining reasonable image data acquisition frequency based on the historical data of specific illness state and disease progress of a patient, excessive repeated examination can be avoided, burden and radiation exposure of the patient are reduced, and expenditure of acquisition cost is reduced, definition of images can be remarkably improved through denoising process, random errors (such as artifacts in MRI, ray noise in CT and the like) generated by noise in imaging equipment are removed, boundaries and details of focus can be clearly shown on the denoised images, gray scale standardization can standardize image gray values of different equipment or different time points to a uniform range, consistency is achieved among the images, irrelevant areas in the images can be removed through image clipping processing, and data redundancy is reduced.
Further, rigid registration of lesion image data by affine transformation, comprising:
based on the preprocessed image data, performing preliminary alignment on the image data of multiple time points through affine transformation, defining an affine transformation matrix, and calculating a new position of each pixel through matrix multiplication and vector addition operation to perform rigid registration, wherein the rigid registration is expressed as follows: Wherein the method comprises the steps ofRepresenting the post-change coordinates of the pixel coordinate points,Representing the pixel coordinates of the original image,And b is an affine transformation matrix and a displacement vector, respectively, expressed as:; an affine transformation matrix of 3*3 for expressing rotation and scaling, each matrix elementSpecifically defined as control items of rotation, scaling or cutting operations, the rotation angle of the image on each axis is acquired by using metadata of the image (such as pose information at the time of imaging) or by an image analysis tool, a scaling factor is acquired by comparing pixel resolutions of the image,、AndFor displacement values along x, y and z axes in the displacement vector, determining the position difference of the same feature point at different time points through an image analysis tool (such as an automatic feature point detection algorithm SURF), so as to calculate a translation vector;
Determining a reference image based on the training set, selecting a Mean Square Error (MSE) as a similarity measure function, the objective being to minimize the difference between the target image and the reference image, expressed as: WhereinRepresenting the error of the calculation,Representing the total number of pixels,Representing the coordinates of the i-th pixel point,The coordinates of the corresponding pixel points in the reference image;
Performing iterative optimization through an Adam optimizer and a gradient descent method, stopping iteration when the loss calculated in the continuous iterative process is not obviously reduced, determining optimal A and b, and outputting an optimal affine transformation matrix;
Transforming the coordinates of each pixel point to new position coordinates by rigid registrationAn alignment image is generated.
Through rigid registration based on affine transformation, tiny differences of patient postures in the medical image shooting process can be compensated, so that image data of different time points are aligned in space, errors caused by position and angle changes are eliminated, subsequent analysis can be concentrated on changes of focuses, the changes are not influenced by deviations in image acquisition, and a reliable data base is provided for subsequent focus volume changes, morphological change trend predictions and the like.
S2, predicting focus probability distribution based on a depth convolution neural network, and performing focus segmentation and marking based on a Conditional Random Field (CRF);
Preferably, predicting the lesion probability distribution based on the deep convolutional neural network, and performing lesion segmentation and marking based on the conditional random field CRF, including:
Constructing a focus area model based on a depth convolutional neural network, wherein the focus area model comprises an input layer, a convolutional layer, a maximum pooling layer, a full-connection layer and an output layer;
the input layer inputs the aligned images subjected to rigid adaptation, the convolution layer extracts local features by scanning local regions of the images, the maximum pooling layer reduces the size of the feature images and highlights the significance of the features, the full-connection layer maps the features to the output layer, and the output layer outputs prediction results of lesion region segmentation;
using a training set training model, selecting a cross entropy loss function to calculate the difference between a focus area model prediction result and an actual label, using an Adam optimizer to perform gradient descent optimization, updating parameters of the model, and stopping iterating the output model when the loss of the model is not obviously reduced in a continuous iterating process;
Obtaining a focus probability distribution map based on the alignment image based on a focus region segmentation prediction result output by the focus region model;
based on the conditional random field CRF, the probability that each pixel belongs to the lesion area is taken as a single-point term, the similarity between the pixel points is defined as a double-point term, and an energy function is defined based on the single-point term and the double-point term, and is expressed as:;; WhereinIndicating the cost to which the ith pixel belongs to the lesion,Representing the probability that the ith pixel belongs to the lesion area,Representing the similarity cost of adjacent i-th and j-th pixels,AndRepresenting the spatial coordinates of the i-th pixel and the j-th pixel respectively,The weight coefficient is represented by a number of weight coefficients,AndRespectively representing the gray values of the ith pixel and the jth pixel,AndRespectively representing a spatial scale parameter and a color scale parameter,Represents the total energy function value, N represents the total number of pixel points,Representing the adjacent pixel area of the ith pixel,Representing the Euclidean distance of the ith pixel and the jth pixel;
Calculating loss between lesion prediction result and actual label of conditional random field CRF by using training set and selecting cross entropy loss function, gradient descent optimizing by using Adam optimizer, updating conditional random field CRF parameters including、AndStopping iteration when the calculated loss is not obviously reduced in the continuous iteration process, outputting model parameters, and updating the model;
Optimizing an energy function by using a Mean-Field Approximation Mean field approximation method, and determining an optimization formula, wherein the optimization formula is expressed as follows: WhereinThe normalization factor is represented as such,The probability of classification is represented by a number of classes,Representing a classification probability of neighboring pixels;
taking a prediction probability map of a focus area model as an initial valueDetermining;
Based onUpdating the classification probability of each pixel as in successive iterationsStopping iteration when the variation of the model is no longer obviously reduced, and outputting the final model;
Based on the final resultGenerating a corresponding focus area probability map, dividing pixel values of the probability map based on the historical average value as a dividing threshold value, and marking the focus area.
By using the deep convolutional neural network, local features can be extracted based on the scanned image, important information such as edges, shapes and textures of a focus area can be learned, the size of a feature map is reduced through a maximum pooling layer, the calculation burden is reduced, the important information of the features is reserved, the image features can be mapped to a probability space through a full-connection layer and an output layer by a model, the prediction probability distribution of the focus area is generated, and the basic identification according to the scanned image is realized;
Modeling the focus probability of each pixel and the similarity cost of adjacent pixels based on defining a single-point term and a double-point term through the use of a Conditional Random Field (CRF), and matching with the optimization of a mean value field approximation method, by gradually updating the classification probability of each pixel, each pixel considers not only the prediction result of the pixel but also the influence of the adjacent pixels, so that the classification of the pixel is more robust, the problem of classification error caused by local noise in an image is effectively solved, and the probability distribution map of a focus area can be gradually optimized through repeated iterative updating, so that the segmentation result is finer and more accurate;
by combining the deep convolutional neural network and a Conditional Random Field (CRF), the accuracy of focal region segmentation is improved, the human intervention is reduced, the method can adapt to complex focal morphology, the focal can be marked efficiently and accurately, and a reliable basis is provided for subsequent diagnosis and treatment planning.
S3, describing local displacement by using a B-spline basis function to perform non-rigid registration, and generating a final image;
preferably, the non-rigid registration of local displacements using B-spline basis function descriptions generates a final image, comprising:
Generating grids based on focus areas of the image data, defining grid nodes as control points, and performing non-rigid registration;
the local displacement is described using a B-spline basis function based on each control point, expressed as:; WhereinRepresents the displacement amount of the pixel x, n represents the number of control points adjacent to the pixel,The displacement vector representing the control point i,The B-spline basis function is represented,Representing the coordinates of the i-th control point,Representing a third-order B-spline basis function, expressed as: Determining a reference image based on the training set, selecting a mean square error MSE as a similarity measure function, wherein the aim is to minimize the difference between the target image and the reference image, expressed as: WhereinRepresenting the non-rigid registration calculation error,Representing the total number of pixels,Representing the coordinates of the i-th pixel point,For the corresponding pixel point coordinates in the reference image, M represents the total number of control points,Represents a smoothness regularization parameter, which is determined based on historical data,Representing the gradient of the displacement vector of the control point j,Representing pixel pointsDisplacement of (2);
Displacement according to current control point using Levenberg-Marquardt optimization algorithmCalculating the value of the non-rigid registration calculation error, and calculating the displacement of the objective function for each control pointThen updating the control point displacement, stopping iteration when the calculated error in the continuous optimization process is not obviously reduced any more, and optimizing the displacement of the control point;
Calculating pixel point displacement based on control point displacement by non-rigid registrationAnd further adjusting the positions of the pixel points based on the aligned images to generate a final image.
The grid is generated in the focus area, grid nodes are defined as control points, a basis is provided for non-rigid registration, local deformation of the focus area can be accurately controlled, displacement of pixels can be smoothly interpolated between the control points through a B-spline basis function, the change of the local displacement is smooth and free of mutation, artifacts or unnatural edges which possibly occur in registration results are avoided, small morphological changes in the focus area, such as fine deformation of the focus edge, can be captured in the registration process through a third-order B-spline basis function, the non-rigid registration can be aligned with the boundary of the focus area on local details, higher segmentation accuracy is provided, and the optimal displacement vector of the control point can be quickly and effectively found through the advantages of gradient descent and Newton method by a Levenberg-Marquardt optimization algorithm;
The displacement of the pixel points is calculated through the displacement of the control points, each pixel point can be ensured to be accurately moved to a new position, the alignment of a focus area with higher precision is realized, particularly when the morphology of the focus area is greatly changed at different time points, the non-rigid registration can accurately capture the changes, the morphology and the position of the focus area can be accurately aligned by the generated image after the image is globally and locally adjusted based on the displacement of the control points, reliable basic data are provided for the follow-up focus change analysis, morphology change quantification and other works, and the precision and consistency of the image analysis can be obviously improved;
The model-identified lesion area probability map provides an accurate initial position of a lesion for registration, but non-rigid registration further improves the identification accuracy through local adjustment, the collaborative optimization is not realized by a single technology, especially in complex lesion morphology change, the combined effect can obviously reduce errors, meanwhile, the combination of rigid registration and non-rigid registration greatly improves the stability of registration, the problem of global instability possibly caused by purely depending on non-rigid registration is avoided, through the combination of rigid registration and model identification, the global alignment of images can be quickly realized, the change of a lesion area can be carefully tracked, especially for the fine change of lesion expansion or contraction in the disease progress process, a reliable initial position can be provided for the follow-up local change, the accurate alignment and analysis of a lesion area in the fine morphology change can be ensured, the doctor can be assisted in tracking the disease condition of a patient through the change of image data, especially for a patient who is treated for a long time, the doctor can analyze the treatment effect through the image change analysis, the treatment adjustment can be automatically carried out, the accurate image analysis can be greatly reduced, the intelligent image analysis can be realized through the intelligent analysis of the lesion area, and the accurate analysis can be more improved.
S4, selecting a focus area to perform volume calculation, and determining a focus volume increase rate and a focus volume change rate by using a nonlinear differential equation;
Preferably, the focal region is selected for volumetric calculation, including,
Based on the image scanning equipment, three-dimensional image data are formed through stacking of multiple layers of image data, a focus change area is determined based on a final image generated through non-rigid registration, voxels of the focus change area are selected for volume calculation, and the three-dimensional image data are expressed as: wherein V represents the volume of the lesion area,、AndThe length, width and height of the voxels in the three-dimensional space are respectively represented, i, j and k are respectively indexes of the voxels, and the voxels in all focus areas on the three-dimensional grid are represented.
The volume is calculated through the voxels, so that fine and reliable focus volume measurement can be provided, especially for the condition of complex focus morphology or blurred focus edge, the real space distribution of the focus can be captured, the three-dimensional volume can be more accurately analyzed through non-rigid registration and voxel volume calculation, the disease development of a patient can be dynamically tracked, especially in the treatment process, the change condition of the focus volume can be periodically estimated, and a doctor can be helped to make more accurate judgment.
Further, a non-linear differential equation is used to determine the rate of lesion volume increase and rate of change, including,
Fitting the volume change data of the lesion using a nonlinear differential equation is expressed as: WhereinThe lesion volume at time t is indicated,Indicating the rate of increase of the lesion volume,A maximum limit value representing a lesion volume;
Calculating a minimized difference between the observed volume and the predicted volume using the historical data and using a square error (MSE) as a loss function;
Calculating loss by using nonlinear least square method, stopping iteration when the loss of the model is not obviously reduced in the continuous iteration process, and outputting final K and K;
The rate of change of the growth rate is determined based on the change of the lesion volume growth rate at different times, and the rate of change is evaluated according to the experience of the physician.
By utilizing historical focus volume data and fitting model parameters through a nonlinear least square method, the method can accurately predict the volume change of a focus at future time, can help doctors predict the development trend of the focus, is convenient for making intervention measures in advance, evaluates the expansion or shrinkage condition of the focus in a future period of time, achieves early warning function of the disease condition, reduces the risk of patients and improves the prognosis effect.
Embodiment 2 referring to fig. 2, a system for a patient condition tracking method is provided for a second embodiment of the present invention, which is different from the previous embodiment, and includes:
The image data processing module is used for collecting focus image data of a patient and carrying out image preprocessing;
a rigid registration module for rigid registration by affine transformation;
the focus recognition and segmentation module is used for predicting probability distribution of focus areas based on the depth convolution neural network and carrying out focus segmentation and marking by using a conditional random field;
The non-rigid registration module is used for describing local displacement through the B-spline basis function and carrying out non-rigid registration;
The focus volume calculation module is used for selecting a focus area to perform volume calculation based on the image data after non-rigid registration;
a volume change analysis module for determining a volume increase rate and a change rate of the lesion using a nonlinear differential equation.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium include an electrical connection (an electronic device) having one or more wires, a portable computer diskette (a magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of techniques known in the art, discrete logic circuits with logic gates for implementing logic functions on data signals, application specific integrated circuits with appropriate combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
It should be noted that the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted without departing from the spirit and scope of the technical solution of the present invention, which is intended to be covered in the scope of the claims of the present invention.