WO2023128974A1 - A system and method for detecting the position of a robotic capsule endoscope with permanent magnet inside the body - Google Patents

Abstract

The invention relates to a system for detecting the position of robotic capsule endoscopes (E) with permanent magnet inside the body, comprising magnetic field sensors for being located around a body and measuring the magnetic flux densities generated by a robotic capsule endoscope with permanent magnet inside the body; a processing unit adapted to execute an optimization algorithm that determines a position in a selected working space, which gives measured magnetic flux densities, as a first position of a robotic capsule endoscope with permanent magnet; wherein the processing unit is adapted to calculate a working area (1) in the workspace containing possible change positions of the robotic capsule endoscope with permanent magnet with respect to the previous position in the unit position determination time for determining the next positions of the robotic capsule endoscope with permanent magnet, and to execute the optimization algorithm that determines a position in the working area (1), which gives the next measured magnetic flux densities, as the next position of the robotic capsule endoscope with permanent magnet.

Description

A SYSTEM AND METHOD FOR DETECTING THE POSITION OF A ROBOTIC CAPSULE ENDOSCOPE WITH PERMANENT MAGNET

INSIDE THE BODY

Field of the Invention

The present invention relates to a system and method for detecting the position of a robotic capsule endoscope inside the body.

Background of the invention

The medical imaging method that allows the examination of the gastrointestinal (GI) system, which includes the esophagus, stomach, duodenum and large intestine, by means of a device called endoscope containing a small lighted camera, is called endoscopy. The endoscope probes, which started to be used especially after the fiber optic techniques became widespread, have become the most vital tool used in the diagnosis of gastrointestinal diseases. Endoscopy is widely used for diagnostic examinations and medical imaging procedures for human and animal patients, especially for conditions such as small bowel disorders, colon cancer, Crohn's disease. A classical endoscopy probe consists of a flexible and long tube/cable with a miniature camera at its end and a lighting system. The desired area of the digestive system is accessed by entering through the mouth or anus with the tube/cable tip and this area is visualized by means of a camera. However, due to the large diameter and certain rigidity of the tube/cable, endoscopy can be painful for patients, depending on the dexterity of the specialist performing the operation. Even development of bleeding and infection in the intestine can also be observed in some cases. Moreover, classical endoscopy cables can reach an average depth of 2.5m in the intestine. Considering that the average length of the human intestine is 9 meters, endoscopies performed with classical endoscopes are insufficient for complete visualization of the intestine. To overcome the aforementioned drawbacks and limitations of classical endoscopy, Wireless Capsule Endoscopy (WCE) has been introduced as an alternative to wired endoscopy. Capsule endoscopy is a non-invasive imaging method that allows observation of regions of the GI tract that cannot be reached by classical endoscopy methods and diseases in these regions. The capsule typically has the size of a large pill and contains a camera, a lighting unit (e.g. an LED), a processing unit, a battery and a wireless communication unit (e.g. an RF antenna). The capsule, which is taken orally into the body, can travel actively or passively through the gastrointestinal tract. While traveling, it sends images or videos of the digestive system wirelessly to the external environment. The fact that capsule endoscopy allows painless and non-invasive observation not only of the intestine but also of the entire GI tract, that it requires no wiring and that is patient-friendly undoubtedly brings this technique to the forefront in the diagnosis of digestive system diseases.

The capsule endoscopes whose movement in the GI tract can be controlled externally are called robotic capsule endoscopes. Thanks to robotic capsule endoscopy, the endoscope can be used not only for diagnostic purposes but also for therapeutic purposes. In robotic capsule endoscopy, the location and angles of the robotic capsule endoscope in the GI tract must be known in order to guide the capsule endoscope to the desired location. Precisely knowing the position and orientation of the robotic capsule endoscope moving in the GI tract is important for determining which region of the GI tract the endoscopic images belong to. Because, without this determination, it is not possible for the endoscopist to evaluate the endoscopic images correctly. Considering that the GI tract has an average length of 9 meters, it is not enough to detect the presence of a disease in this tract, but it is also necessary to know the exact location of the disease in the intestine for the specialist who will perform the operation. Therefore, the follow-up of the effects of a drug treatment, the follow-up of the intervention and the treatment operations will depend mostly on the accuracy and precision of the location information of the endoscopic images. The main reason why classical endoscopy methods are still widely used today instead of robotic capsule endoscopy is because the localization of the robotic capsule endoscope inside the human body cannot be done effectively. Since said localization cannot be performed, capsule endoscopy passively travels inside the body and can only be used for diagnostic purposes. In the case that the robotic capsule endoscope can be positioned inside the body with the desired precision (location and angle of the capsule endoscope) and thus controlled from outside the body, it will be possible to use robotic capsule endoscopy for diagnosis as well as purposes such as treatment, drug release, specimen sampling (biopsy), etc.

In the prior art, there are studies on image processing, RFID, RSSI, Electromagnetic waves (RF, Visible spectrum, X-rays, Gamma-rays), Ultrasound, MRI and magnetic field based methods for detecting the position of the robotic capsule endoscope in the GI tract. These methods are compared in Table 1.

Table 1. Methods for detecting the position of robotic capsule endoscopy in the prior art.

Among the methods for detecting the position of the robotic capsule endoscope, X- ray and gamma-ray methods are the ones with the lowest position detection error, however, X-rays and gamma rays are harmful to the human body. Therefore, the use of these methods has the potential of causing health problems. Although RF methods, another position detection method, can be used successfully in different areas such as indoor and outdoor position detection, position detection errors are quite high in complex body environment. In RF position detection methods, the environment must be pre-modeled. The fact that the tissues in the human body have different electrical properties and the tissue location and size vary from person to person is the biggest obstacle in creating a fixed body model. Therefore, there are problems that need to be solved for the use of RF methods for detecting the position of the robotic capsule endoscope.

As can be seen in Table 1, the permanent magnet method is one of the most studied methods for robotic capsule endoscopes because it is suitable for real-time operation, harmless to the human body and its accuracy is more precise than other techniques. In the permanent magnet method, a permanent magnet, which is usually in the form of a ring, is used such that it surrounds the capsule. The magnetic flux density generated by the magnet is detected by sensors located around/on the body, thus determining the position of the robotic capsule endoscope in the GI tract. In the studies of the prior art, optimization algorithms using magnetic flux density data from sensors are utilized to detect the position of the robotic capsule endoscope. In these studies, the space in which the optimization algorithms will run is a predetermined fixed space. A position is determined for the robotic capsule endoscope in the said predetermined fixed space, which will provide the magnetic flux density data by optimization algorithms. In other words, the optimization algorithm scans the space and a position in the space that will provide the magnetic flux density data is determined. The predetermined fixed space in which the optimization algorithms will run is re-used each time that the position detection of the robotic capsule endoscope is performed. That is, for detecting the next position of the robotic capsule endoscope, new magnetic flux density data is received from the sensors, and the predetermined fixed space is re-used to determine the new position that will provide the new flux density data in this space. In systems or methods in which this use is performed, the position detection of the robotic capsule endoscope cannot be performed with the desired precision.

As a result, there is a need in the art for a system and a method for detecting the position of the robotic capsule endoscope inside the body in order to effectively perform robotic capsule endoscopy.

Detailed Description of the Invention

For a better understanding of an embodiment of a system for detecting the position of robotic capsule endoscopes with permanent magnets inside the body, which is realized to achieve the objective of the present invention, the embodiment is shown in the accompanying figures. The details of the invention should be evaluated taking into consideration the entire description. In the figures:

Figure 1. is a schematic view of the calculated working area for determining the position of a robotic capsule endoscope with permanent magnet moving through the small intestine in an embodiment of the invention.

Figure 2. is a graph depicting the location of the robotic capsule endoscope and magnetic field sensors associated with the small intestine in an embodiment of the invention.

Figure 3. is a graph depicting the effect of radius size of the working area on determining the position of the robotic capsule endoscope in one embodiment of the invention.

Figure 4. is a comparative graph depicting the robotic capsule endoscope position accuracies determined by a prior art method and by the system/method of the present invention using a working area radius of 14mm in one embodiment of the invention.

Figure 5. is a comparative graph depicting the robotic capsule endoscope position accuracies determined by a prior art method and the system/method of the present invention in the presence of AWGN noise in an embodiment of the invention.

The elements shown in the figures are each given reference numbers as follows:

1. Working area

E. Robotic capsule endoscope with permanent magnet r_s. Radius of the working area in. an instant location of the robotic capsule endoscope in-i. A previous location of the robotic capsule endoscope

B: Small intestine

A system for detecting the position of robotic capsule endoscopes (E) with permanent magnet inside the body comprises:

- magnetic field sensors for being located around a body and measuring the magnetic flux densities generated by a robotic capsule endoscope with permanent magnet inside the body; - a processing unit adapted to execute an optimization algorithm that determines a position in a selected working space, which gives measured magnetic flux densities, as a first position of a robotic capsule endoscope with permanent magnet;

- wherein the processing unit is adapted to calculate a working area (1) in the workspace containing possible change positions of the robotic capsule endoscope with permanent magnet with respect to a previous position in the unit position determination time for determining the next positions of the robotic capsule endoscope with permanent magnet, and to execute the optimization algorithm that determines a position in the working area (1), which gives the next measured magnetic flux densities, as the next position of the robotic capsule endoscope with permanent magnet.

In the embodiments of the invention, the position of the robotic capsule endoscope with permanent magnet inside the body is determined. The position herein is the location of the capsule endoscope in a coordinate plane and the rotation angles (yaw, pitch, roll) in three dimensions about the center of mass of the capsule endoscope. The initial position of the robotic capsule endoscope with permanent magnet inside the body is determined by being calculated at a position in the working space by an optimization algorithm. Here, the optimization algorithm answers the question at which position in the working space the capsule endoscope should be so that the measured magnetic flux densities can be obtained. This answer defines the initial position of the robotic capsule endoscope inside the body at the moment when the magnetic flux densities are measured. The working space can be of a size to include all or part of the GI tract of the body, for example the small intestine (B), where the robotic capsule endoscope can be located. When the robotic capsule endoscope is moved, the endoscope moves to the next position in the working space. The magnetic flux densities are measured when the endoscope is in the said next position (next measured magnetic flux densities). In order to determine the next position of the endoscope, the optimization algorithm must calculate and determine the position of the endoscope in the working space that gives the next measured magnetic flux densities. However, calculating the changing robotic capsule endoscope position each time in a working space that can be of the said size increases the margin of error of the determined position. Therefore, in the invention, once the initial position of the robotic capsule endoscope is determined, the next positions thereof are determined by a working area (1) adjacent to the initial position within the working space, instead of the working space. The geometry of the working area (1) varies depending on the robotic capsule endoscope speed and the positioning period (unit position determination time) of the system. Thus, the optimization algorithm calculations for determining the positions of the robotic capsule endoscope subsequent to its initial position are performed in a space much smaller than the working space (working area ((1)). This shortens the time it takes for the optimization algorithm to answer the said question and increases its accuracy. In the case studies carried out, measurement systems in which all position determinations are performed in the working space were compared with measurement systems in the working area (1) of the invention. In this comparison, with the system or method of the present invention, the position of the robotic capsule endoscope was determined with a lower margin of error of 72.59% in location and 73.58% in angle. Furthermore, it has been observed that the position determination performed in the working area (1) of the present invention is affected less by magnetic noise.

An embodiment of the invention comprises magnetic field sensors calibrated for eliminating noise magnetic flux values originating from the earth and the measurement environment.

An embodiment of the invention comprises optimization algorithm which is Artificial Bee Colony.

An embodiment of the invention comprises a processing unit adapted to calculate the position of robotic capsule endoscope (E) with permanent magnet, which gives the measured magnetic flux densities, by running iteratively until it meets a convergence criterion selected by using a magnetic flux model.

An embodiment of the invention comprises a processing unit adapted to recalculate the determined position of the robotic capsule endoscope with permanent magnet by selecting the determined position values of the robotic capsule endoscope with permanent magnet as initial values of a Levenberg-Marquart algorithm, for example running with a Gauss-Newton method, that enables solving nonlinear problems by least squares method.

A method for detecting the position of robotic capsule endoscopes (E) with permanent magnet inside the body comprises the following steps:

- measuring the magnetic flux densities generated by a robotic capsule endoscope with permanent magnet inside the body;

- determining a position in a selected working space, which gives measured magnetic flux densities, as an initial position of a robotic capsule endoscope with permanent magnet;

- calculating a working area (1) in the working space containing the possible change positions of the robotic capsule endoscope with permanent magnet with respect to the previous position in the unit position determination time for determining the next positions of the robotic capsule endoscope with permanent magnet;

- determining a position in the working area (1), which gives the next measured magnetic flux densities, as the next position of the robotic capsule endoscope with permanent magnet.

An embodiment of the invention comprises the step of eliminating the noise magnetic flux values originating from the earth and the measurement environment while the magnetic flux densities generated by the robotic capsule endoscope with permanent magnet are measured. In an embodiment of the invention, the method comprises the step of calculating the position of robotic capsule endoscope (E) with permanent magnet that gives the measured magnetic flux densities, by running iteratively until it meets a convergence criterion selected by using a magnetic flux model.

An embodiment of the invention comprises the step of measuring the position of the robotic capsule endoscope with permanent magnet, which gives the measured magnetic flux densities, with an Artificial Bee Colony optimization algorithm.

An embodiment of the invention comprises the step of recalculating the determined position of the robotic capsule endoscope with permanent magnet by selecting the determined position values of the robotic capsule endoscope with permanent magnet as initial values of a Levenberg-Marquart algorithm, for example running with a Gauss-Newton method, that enables solving nonlinear problems by least squares method.

The previous position of the endoscope is utilized while determining the positions of the robotic capsule endoscope with permanent magnets subsequent to its initial position. The working area (1) can be defined in a Cartesian, spherical or cylindrical coordinate system. In an exemplary embodiment of the invention, the working area (1) is determined as follows. In this embodiment, the working area (1) is defined in the spherical coordinate system (See Figure 1). The previous location (i_n-i) of the robotic capsule endoscope is utilized while calculating the instant location (i_n) of the robotic capsule endoscope (desired location to be determined). Here, the vector in defines the n^th location (x_n, y_n, z_n) of the robotic capsule endoscope. The radius r_s defines the spherical working area (1) in which the robotic capsule endoscope can be located instantaneously, with its center at the previous location (i_n-i) of the robotic capsule endoscope. The radius (r_s) of the working area (1) is determined as the maximum Euclidean distance between each determined position of the robotic capsule endoscope. In other words, the radius of the working area (r_s) is determined depending on the robotic capsule endoscope speed and the position determination period (unit position determination time) of the system. Thus, a radius (radius of the working area (r_s)) is determined to form a working area (1) that will cover all the positions that the capsule can take during the time between two position determinations of the robotic capsule endoscope system. It is important to determine the radius (r_s) of the working area correctly. In the event that the radius of the working area (r_s) is set too large, it causes errors in the next determined position of the robotic capsule endoscope that converge to the prior art. In the event that the radius of the working area (r_s) is set too small, the next position of the robotic capsule endoscope is again incorrectly determined. This error increases cumulatively for each next position determination.

In this embodiment, the working space used to determine the initial position of the robotic capsule endoscope can be defined as follows: 0”,0”,0°]

, 180° , 360” , 360° ]

Equation 1 wherein (x^,^,^^) is the minimum coordinates of the positions in which the robotic capsule endoscope can be located when determining the initial position of the robotic capsule endoscope and (x^,}^,^) is the maximum coordinates of the positions in which the robotic capsule endoscope can be located.

In the prior art, Equation 1 is also used to determine the next position of the robotic capsule endoscope. That is, the geometry of the working space and the working area (1) are identical/equal.

However, in the invention, the next position of the robotic capsule endoscope is determined using the following working area (1):

Equation 2 wherein i_K-1 is the previous location vector of the robotic capsule endoscope, r_s is the radius of the working area (r_s).

As seen in Equation 2, the working area (1) in which the next position of the robotic capsule endoscope is determined adaptively changes based on the position of the previous robotic capsule endoscope.

In order to assess the effectiveness of the invention, various studies have been carried out. In these studies, a working space for a small intestine (B) model was determined. Six magnetic field sensors were used in this study. These magnetic field sensors were placed in the umbilical region of the body. 317 different positions of the robotic capsule endoscope with permanent magnet in the small intestine (B) were determined by the system/method of the present invention and a method of prior art. According to this determination data, the prior art and the system/method of the present invention were compared in terms of position determination accuracy. In this study, the permanent magnet of the robotic capsule endoscope was a Neodymium Iron 42 (N42) permanent magnet with 15mm outer diameter, 10mm inner diameter and 6mm length. The magnetization value of the permanent magnet was 430.000 (A/m) in the diameter direction. It was assumed that the robotic capsule endoscope was rotated in the axes X, Y, and Z by the angles (a,p,y) respectively, and as a result, the 6D locations and angles (x_m,_m,z_m,oc,p,y) of the permanent magnet (i.e. the robotic capsule endoscope) were determined/calculated by the magnetic flux model.

Due to the fact that the magnetic flux model is nonlinear, the known location of the magnetic field sensors and the magnetic flux density values measured by the sensors were executed in the optimization algorithm. Minimization process was earned out with the least squares technique. Artificial Bee Colony (ABC) algorithm and Levenberg -Marquardt (LM) algorithm were executed in a hybrid manner as the optimization algorithm. It was aimed to minimize the objective function in Equation 3 below in both algorithms.

Equation 3

(3 B B wherein,^s denotes the error value, N is the number of sensors,^y' ’^z \.

(B B B } denotes the magnetic flux density values measured by the sensor and '^x‘ ’^y' ’^z' denotes the magnetic flux density values estimated by the optimization algorithms. As a result, 6 unknown parameters ( A AA

_robotic capsule endoscope were obtained which provide the magnetic flux density values obtained from the magnetic field sensors. In order to demonstrate the localization and angle performance (position determination performance) of the system, Equation 4 below was calculated at each iteration.

Equation 4

In Equation 4, (x_m,y_m,z_m,y, p,a) represents the known actual location metrics of the robotic capsule endoscope, (x_m,y_m,z_m,f,P,a) represents the location metrics determined by the system/method of the present invention, E_p represents the position error, and E_o represents the angle error.

Determinations were made for 317 positions of the robotic capsule endoscope in the small intestine (B). 100 runs were performed, and the mean location and angle errors were obtained with equation 2. The population of the optimization algorithm (ABC) was determined to be 30 and the number of iterations to be 45. The effect of the size of the radius of the working area (r_s) on determining the positions of the robotic capsule endoscope was investigated (see figure 3).

Considering the possible locations of the robotic capsule endoscope in the small intestine (B) and the distances between these locations, the maximum Euclidean distance between locations was determined to be 14 mm. Similarly, it has been seen in Figure 4 that the highest position determination accuracy is obtained at 14 mm working area radius (r_s) value. The accuracy of the positions determined by a method of the prior art and the accuracy of the positions determined by the inventive system/method using a working area radius (r_s) value of 14 mm are compared in figure 4 and table 2.

Table 2

When the results in all comparison parameters are analyzed, it has been observed that the position of the robotic capsule endoscope is determined more accurately with the system/method of the present invention compared to the prior art. With the system/method of the present invention, it has been observed that the margin of error in determining the position of the robotic capsule endoscope has been improved by at least 40% compared to the prior art and by more than 72% on average.

The performance of the system/method of the present invention was also investigated under noisy measurement conditions. Although magnetic noise can be reduced by calibrating magnetic field sensors, it is not possible to eliminate noise completely. For this reason, the accuracy of the positions determined by the system/method of the present invention was also tested with a noise [Additive White Gaussian Noise (AWGN)] added to the magnetic field sensor measurements. Here, the performance measurement was performed for the noise values which range from 20dB to 40dB, and which significantly distort the magnetic flux density measured by the magnetic field sensors, for 61 different values of Signal to Noise Ratio (SNR) and is shown in Figure 5.

When the noise-added simulation results shown in Figure 5 are analyzed, it is seen that determining the position of the robotic capsule endoscope with the system/method of the present invention is about 27.6% better in location error and about 36.35% better in angle error compared to the prior art. As a result, in the case study, compared to the prior art, a significant improvement in the margin of error in determining the position of the robotic capsule endoscope has been achieved with the system/method of the present invention in both noisy and noiseless measurement conditions.

Claims

CLAIMS A system for detecting the position of robotic capsule endoscopes (E) with permanent magnet inside the body, comprising magnetic field sensors for being located around a body and measuring the magnetic flux densities generated by a robotic capsule endoscope with permanent magnet inside the body; a processing unit adapted to execute an optimization algorithm that determines a position in a selected working space, which gives measured magnetic flux densities, as a first position of a robotic capsule endoscope with permanent magnet, characterized in that the processing unit is adapted to calculate a working area (1) in the workspace containing possible change positions of the robotic capsule endoscope with permanent magnet with respect to the previous position in the unit position determination time for determining the next positions of the robotic capsule endoscope with permanent magnet, and to execute the optimization algorithm that determines a position in the working area (1), which gives the next measured magnetic flux densities, as the next position of the robotic capsule endoscope with permanent magnet. System according to claim 1, comprising magnetic field sensors calibrated for eliminating noise magnetic flux values originating from the earth and the measurement environment. System according to any one of claims 1 -2, comprising optimization algorithm which is Artificial Bee Colony. System according to any one of claims 1-3, comprising a processing unit adapted to execute the optimization algorithm that calculates the position of the robotic capsule endoscope (E) with permanent magnet, which gives the measured magnetic flux densities, by running iteratively until it meets a convergence criterion selected by using a magnetic flux model. System according to any one of claims 1-4, comprising a processing unit adapted to recalculate the determined position of the robotic capsule endoscope with permanent magnet by selecting the determined position values of the robotic capsule endoscope with permanent magnet as initial values of a Levenberg-Marquart algorithm that enables solving nonlinear problems by least squares method. A method for detecting the position of robotic capsule endoscopes (E) with permanent magnet inside the body, comprising the following steps:

- measuring the magnetic flux densities generated by a robotic capsule endoscope with permanent magnet inside the body;

- determining a position in a selected working space, which gives measured magnetic flux densities, as an initial position of a robotic capsule endoscope with permanent magnet;

- calculating a working area ( 1) in the working space containing the possible change positions of the robotic capsule endoscope with permanent magnet with respect to the previous position in the unit position determination time for determining the next positions of the robotic capsule endoscope with permanent magnet;

- determining a position in the working area (1), which gives the next measured magnetic flux densities, as the next position of the robotic capsule endoscope with permanent magnet. Method according to claim 6, comprising the step of eliminating the noise magnetic flux values originating from the earth and the measurement environment while the magnetic flux densities generated by the robotic capsule endoscope with permanent magnet are measured. Method according to any one of claims 6-7, comprising the step of calculating the position of robotic capsule endoscope (E) with permanent magnet, which gives the measured magnetic flux densities, by running iteratively until it meets a convergence criterion selected by using a magnetic flux model. Method according to any one of claims 6-8, comprising the step of measuring the position of the robotic capsule endoscope with permanent magnet, which gives the measured magnetic flux densities, with an optimization algorithm which is Artificial Bee Colony. Method according to any one of claims 6-9, comprising the step of recalculating the determined position of the robotic capsule endoscope with permanent magnet by selecting the determined position values of the robotic capsule endoscope with permanent magnet as initial values of a Levenberg-Marquart algorithm that enables solving nonlinear problems by least squares method.

18