WO2025057139A1 - System and method for generating magnetic fields using an electromagnet array and a permanent magnet - Google Patents

Abstract

This invention provides a system for controlling a magnetic object in a working space. In one embodiment, said system comprises: a) An actuation unit comprising an electromagnetic array, a robotic manipulator, and an external permanent magnet; b) A localization unit comprising a Hall-effect sensor array and a localization algorithm; and c) A computing unit comprising: i) a processor; ii) memory; and iii) program instructions, stored in the memory, that upon execution by the processor cause the computing unit to perform operations for designing a configuration of said electromagnetic array and/or a configuration of said external permanent magnet to generate a desired magnetic field to control said magnetic object.

Description

Dkt.008-05-A-PCT SYSTEM AND METHOD FOR GENERATING MAGNETIC FIELDS USING AN ELECTROMAGNET ARRAY AND A PERMANENT MAGNET FIELD OF THE INVENTION [0001] The invention relates to a system and a method for generating magnetic fields to control magnetic objects. BACKGROUND OF THE INVENTION [0002] Magnetic manipulation systems utilize magnetic fields to manipulate and control medical devices or components within the human body. Such systems have the potential to offer precise, minimally invasive procedures and targeted therapies. Several advantages related to magnetic manipulation systems for medical applications are listed as follows: A) Minimally Invasive Procedures: Traditional surgical procedures often involve large incisions and prolonged recovery times. Medical devices that can be manipulated using magnetic fields can offer a less invasive approach, reducing tissue damage, pain, and recovery time for patients. B) Remote Control and Precision: Magnetic actuation systems allow medical professionals to remotely control the movement of devices within the body. This level of precision is particularly crucial when navigating delicate or hard-to-reach areas, such as blood vessels or areas near sensitive organs. C) Avoidance of Radiation: In procedures such as catheterization, fluoroscopy (X-ray imaging) is commonly used to guide the device. However, repeated exposure to X-rays can be harmful. Magnetic actuation systems offer an alternative way to guide devices without relying on ionizing radiation. D) Targeted Drug Delivery: Magnetic nanoparticles attached to drug carriers can be directed to specific locations within the body using external magnetic fields. This enables targeted drug delivery, reducing side effects and improving the effectiveness of treatment. E) Flexible and Adjustable Devices: Magnetic actuation allows for the creation of flexible and adjustable medical devices. For instance, stents or other implants could be adjusted post-implantation without requiring additional surgeries. F) Combination with Imaging Techniques: Magnetic actuation systems can be combined with imaging modalities like MRI for real-time visualization during procedures. This integration enhances the accuracy of device placement and monitoring. [0003] There are a lot of existing magnetic actuation systems that have been developed and applied for various medical applications. These systems utilize magnetic fields to manipulate medical devices, navigate within the body, and enable targeted therapies. Here are a few examples: A) Magnetic Catheter Navigation: Magnetic navigation systems are used to guide catheters and other medical instruments through blood vessels and delicate anatomical structures. These systems typically consist of external magnets that generate controlled magnetic fields. The catheter contains magnetic elements that interact with the external fields, allowing for precise control of its movement within the body. This technology is used for procedures like cardiac ablation, where abnormal heart tissue is treated by delivering controlled energy through a catheter. B) Magnetic Resonance Imaging (MRI)-Guided Interventions: MRI-compatible robotic systems are used to perform minimally invasive procedures within the MRI scanner. These systems utilize non-ferromagnetic materials and components that are safe for use in the strong magnetic field of an MRI machine. Physicians can control the movement of the robotic instruments using magnetic actuation while simultaneously monitoring the procedure in real-time using MRI imaging. C) Magnetic Drug Targeting: Magnetic drug targeting involves attaching magnetic nanoparticles to drug carriers and guiding them to specific target sites using external magnetic fields. This technique is particularly useful for delivering drugs to tumor sites or other specific locations within the body. Magnetic targeting enhances drug accumulation at the target site while minimizing systemic side effects. D) Magnetic Implant Adjustment: In some cases, medical implants like stents or prosthetic devices can be adjusted or repositioned using magnetic fields after they have been implanted. This can reduce the need for additional invasive surgeries to correct implant placement. E) Magnetic Hyperthermia: Magnetic nanoparticles can be injected into tumor tissues and exposed to alternating magnetic fields. The nanoparticles heat up due to magnetic hysteresis losses, leading to localized hyperthermia within the tumor. This approach can be used as a complementary treatment for cancer therapy. F) Magnetic Particle Imaging (MPI): MPI is an emerging imaging modality that utilizes magnetic nanoparticles as tracers. The nanoparticles emit a signal in response to an oscillating magnetic field, which can be used to create high- resolution images. MPI has potential applications in vascular imaging, cell tracking, and molecular imaging. G) Transcranial Magnetic Stimulation (TMS): TMS is a non-invasive technique used for neuromodulation by applying rapidly changing magnetic fields to specific areas of the brain. TMS is used in research and clinical settings to treat conditions such as depression and certain neurological disorders. H) Magnetic Guidewire Navigation: In interventional cardiology and radiology, magnetic guidewires can be used to navigate through complex vascular pathways. External magnetic fields are applied to guide the movement of the guidewire, aiding in precise navigation during procedures. [0004] These examples demonstrate the diverse range of medical applications for magnetic actuation systems. The field continues to evolve with ongoing research and technological advancements, leading to new innovations that enhance medical procedures, diagnostics, and therapies. [0005] Currently, magnetic manipulations systems can be divided into two categories: using a permanent magnet (typically mounted on a robot arm for multi-DOF manipulation) and using an electromagnet array for for multi-DOF manipulation. For magnetic manipulation systems using an electromagnet array, the advantages compared to those using a permanent magnet are listed as follows: A) Precise Control: Electromagnet arrays offer precise control over the strength, direction, and distribution of the magnetic field, enabling accurate manipulation and positioning of objects. B) Adjustability: The magnetic field strength can be adjusted dynamically by varying the current passing through the coils, allowing for real-time adaptability. C) Strong Magnetic Fields: Electromagnet arrays can generate stronger magnetic fields compared to permanent magnets, which can be advantageous for applications requiring greater force or deeper penetration. D) Field Patterns: Electromagnet arrays can create complex magnetic field patterns, such as gradients, oscillating fields, or dynamic field changes, enabling more versatile applications. [0006] While magnetic manipulation systems using a permanent magnet still have some advantages compared to those using an electromagnet array: A) No Power Supply Needed: Permanent magnets do not require an external power source, which simplifies the system design and reduces energy consumption. B) Stability: Permanent magnets provide a constant magnetic field over time without the need for continuous power input or active control. C) Reliability: Permanent magnets have a longer lifespan compared to some electronic components, contributing to the overall reliability of the system. D) Cost-Effectiveness: Permanent magnets can be cost-effective as they eliminate the need for power supply infrastructure and control electronics. E) Simple Integration: Permanent magnets can be integrated into small and compact systems, making them suitable for applications with space constraints. [0007] In summary, the choice between a permanent magnet and an electromagnet array for a magnetic actuation system depends on the specific requirements of the application, including the desired level of control, magnetic field strength, power consumption considerations, system complexity, safety concerns, and space constraints. Both approaches have their own strengths and limitations, and the decision should be based on a thorough evaluation of these factors. It would be a significant advantage to combine both the permanent magnet and electromagnet array so as to to combine the advantages of these two types of systems, which can provide strong and precise magnetic fields to control multi-DOF motion of target magnetic objects. SUMMARY OF THE INVENTION [0008] This invention provides a system for controlling a magnetic object in a working space. In one embodiment, said system comprises: a) An actuation unit comprising an electromagnetic array, a robotic manipulator, and an external permanent magnet; b) A localization unit comprising a Hall-effect sensor array and a localization algorithm; and c) A computing unit comprising: i) a processor; ii) memory; and iii) program instructions, stored in the memory, that upon execution by the processor cause the computing unit to perform operations for designing a configuration of said electromagnetic array and/or a configuration of said external permanent magnet to generate a desired magnetic field to control said magnetic object. BRIEF DESCRIPTION OF THE FIGURES [0009] Figure 1 illustrates the system overview of the magnetic manipulation system for generating magnetic fields according to the features of the proposed invention, which includes: 1. robotic manipulation; 2. external permanent magnet; 3. Hall-effect sensor array; 4. electromagnet array; 5. system frame. [0010] Figure 2A to 2D illustrate cases of the invented configuration with three to six electromagnets forming an upper and lower module. Each electromagnet contains: 4-1: pure iron core; 4-2: copper coil. [0011] Figure 3A and 3B illustrates the workspace of the magnetic manipulation system, with a grid equally separating the workspace. [0012] Figure 4A to 4D illustrate the maximum field at each node of the grid shown in Figure 3 generated by the electromagnet array containing three to six electromagnets shown in Figure 2. [0013] Figure 5A to 5D illustrate magnetic field distribution generated by the electromagnet array containing three to six electromagnets shown in Figure 2 with an unit current passing through each copper coil. [0014] Figure 6A to 6D illustrates four cases of external and internal permanent magnet using regular or Halbach array magnet design. [0015] Figure 7 illustrates a case of permanent magnet design that investigates the force between these two magnets. [0016] Figure 8A and 8B Fillustrates the dragging and levitating force exerted on the internal permanent magnet by the external permanent magnet, which correspond to the four cases shown in Figure 6. [0017] Figure 9 illustrates the working principle of 5-DOF pose estimation of the target magnet object 6 using a Hall-effect sensor array, with 3-1 as the printed circuit board and 3-2 as the Hall-effect sensor. DETAILED DESCRIPTION OF THE INVENTION [0018] The invention relates to a system and a method for generating magnetic fields to control magnetic objects. More specifically, the system involves an electromagnetic array for generating time-varying magnetic fields, an external permanent magnet mounted on a robot arm for generating strong magnetic field gradients, a Hall-effect sensor array for localization feedback of the controlled magnetic object, and a frame for housing and fixing these components. The method consists of a localization algorithm for calculating pose of the magnetic object, a design method for designing the configuration of the electromagnet array and the external permanent magnet, and a control algorithm for generating desired magnetic fields and force to control the magnetic object. [0019] This invention provides a system for controlling a magnetic object in a working space. In one embodiment, said system comprises: a) An actuation unit comprising an electromagnetic array, a robotic manipulator, and an external permanent magnet; b) A localization unit comprising a Hall-effect sensor array and a localization algorithm; and c) A computing unit comprising: i) a processor; ii) memory; and iii) program instructions, stored in the memory, that upon execution by the processor cause the computing unit to perform operations for designing a configuration of said electromagnetic array and/or a configuration of said external permanent magnet to generate a desired magnetic field to control said magnetic object. [0020] In one embodiment, said actuation unit comprises a top layer, a middle layer and a bottom layer; wherein a) said top layer comprises said external permanent magnet mounted on said robotic manipulator; b) said middle layer comprises said working space and said localization unit; and c) said bottom layer comprises said electromagnet array. [0021] In one embodiment, said operations comprise an optimization method for designing said configuration of the electromagnet array. [0022] In one embodiment, said optimization method comprises the steps of: a) Representing^{pose and magnetic moment of each electromagnet in said electromagnetic array with a vector}_{^^(^, ^, ^) and a vector ^^(^, ^); and b) Generating a grid separating said working space} equally with ^ nodes and representing position of each node by a vector ^_^(^, ^, ^) ; c) Representing magnetic field on a specific node generated by an electromagnet to be ^_^(^_^ , ^_^) and a superimposed magnetic field generated by said electromagnet array to be ^_^(^_^); d) Formulating a cost function to maximize said superimposed magnetic field ^_^(^_^) under constraint of applied current as: ^ ₌^argmin_{‖ Eqn (1);}

^^{^} ≤ ^ ≤ ^^{^} e) Storing norm of resultant magnetic field ‖^_^(^_^)‖ at each node of said grid as ^ and average and isotropy index represented as: ^_{^ !} =^{^} " Eqn (2); ;^{f) Selecting a configuration}_{^ .}

^_#$ [0023] In one embodiment, said operations comprise a force-based method for designing said configuration of the external permanent magnet. [0024] In one embodiment, said force-based method comprises the steps of: a) Representing position of said external permanent magnets by a vector ^₆(^, ^, ^), with its magnetic^{moment represented by ^6; b) Representing position of said magnetic object by a vector}_{^7(^, ^, ^), with its magnetic moment represented by ^7; c) Setting} ^₆(^, ^, ^) = ^₇(^, ^, ^ + ℎ) Eqn (4); ^₆ = −^₇ Eqn (5); where h is a constant representing position of said magnetic object located below said external permanent magnet; d) Representing magnetic force exerted on said magnetic object by said external permanent magnet by a vector :(^₆ , ^₇ , ^₆ , ^₇); e) Recording change in magnetic force, : , by moving said magnetic object along axial direction of said external permanent magnet; and f) Determining a configuration for said external permanent magnet by comparing dragging and levitating magnetic force. [0025] In one embodiment, said vector :(^₆ , ^₇, ^₆ , ^₇) is represented by a model selected from a group consisting of magnetic dipole model, magnetic multipole model, fitting model of magnetic force in mathematical paradigm. [0026] In one embodiment, said operations comprise a control algorithm for planning said configuration of the external permanent magnet and currents applied to said electromagnet array to generate desired magnetic fields. [0027] In one embodiment, said control algorithm comprises the steps of: a) Expressing magnetic field and force generated by said external permanent magnet or said electromagnet array using a magnetic dipole model: ^_^ =^;< @3^B_^^B_^⁾ − CD^_^ =^;< ^_^^_^E Eqn (6); :_^ =^F;< 5_=‖^>‖G H^^B_^⁾ + ^B_^^⁾ + (^B_^ ∙ ^)@^ − 5^B_^^B_^⁾DK ^_^ =^F;< 5₌ L_^^_^ Eqn (7); where ^_^, with a magnetic

− ^_O^P$Q6 the displacement vector from said magnetic dipole to the desired position; ^^B_^is the normalized vector of ^_^ (i.e., ^_^^‖^_^^{‖^0}); ^ is a 3×3 identity matrix; ^ is the magnetic moment of the magnetic object to be controlled; b) packing magnetic field and force generated by said external permanent magnets in a matrix form: S^{^ :T = ;< U ^ VX ^ ^ = ](^ )^ Eqn (8); P" 5Y=ZZVZ[ WZ^ZZ U L\X P" P" P"}_{c) determining pose of of said robotic}

manipulator using __P"` = ∏^b ^_c0 _^{^} ^^{^0} =_S^{de P" ^e P"}_{T f 1} Eqn (9); where d involves all the magnetic field

force generated by said electromagnet array in a matrix form: ^ ^{;< ^ V ^0 ⋯ ^ ^k0 ⋯ f p0 T = 3 T ⋮ ⋱ ⋮ n ⋮ n} = ior Eqn (10); where ^k_^

passing through its coil, p_^ represents the current applied to i-th electromagnet; e) Calculating currents applied to said electromagnet array and pose of said external permanent magnet by solving a formulated optimization problem: t ^ −^{^} −^{^} ; where ‖∙‖ represents the

the vector, respectively, ≤ is element-wise (general) inequality symbol. [0028] In one embodiment, said localization algorithm comprises the steps of: a) Measuring a magnetic field ^_xyz{|} generated by said magnetic object using said Hall-effect sensor array and representing said ^_xyz{|} in a stacked matrix form:^{^00 ⋯ 0^} ^_{~ ≜ ^}^{~ ^~} ⋮ ⋱ ⋮ ^^{^, ^ ∈ ℕ}_^^{Eqn (12);} b) Expressing magnetic field said Hall-effect sensory array as a stacked matrix form ^_^

of said magnetic object, ^, and position of ^^ Hall-effect sensor ^_^^: ^^{r(^, ^00) ⋯ ^r(^, ^0^)} ^ _^ ;^{c) Constructing an}

^ ₋ d) Obtaining 5-DOF pose of the magnetic

said optimization function. [0029] In one embodiment, said stacked matrix form ^_^(^, ^_^^) comprises magnetic dipole model, magnetic multipole model, or fitting model of magnetic field in mathematical paradigm. [0030] In one embodiment, said magnetic object comprises an internal permanent magnet configuration selected from regular or Halbach array. [0031] In one embodiment, said external permanent magnet comprises a configuration selected from regular or Halbach array. [0032] In one embodiment, said system further comprises a system frame. [0033] In one embodiment, said Hall-effect sensor array comprises one or more three-axis Hall-effect sensors. [0034] Exemplary embodiments of a capsule endoscope according to the present invention are explained in detail below with reference to the accompanying drawings. The present invention is not limited to the embodiments. [0035] The magnetic manipulation system is used for generating magnetic fields to control magnetic objects, comprising an actuation unit, a localization unit, a design method, a localization algorithm, and a control algorithm, wherein A) the actuation unit is used for generating designed magnetic fields, which contains an electromagnet array, an external permanent magnet, a robotic manipulator, and a frame connecting and fixing these components. B) the magnetic object contains internal permanent magnets and responds to the magnetic fields generated by the actuation unit. C) the localization unit contains a Hall-effect sensor array for measuring magnetic fields generated by the the actuation unit and the the magnetic object. D) the localization algorithm for sensing the pose of the controlled object based on the magnetic fields measured by the the localization unit. E) the design method contains an optimization method for designing the configuration of the electromagnet array and a force-based method for designing the external permanent magnet. F) the control algorithm for planning the pose of the the external permanent magnet and the currents applied to the the electromagnet array to generate desired magnetic fields and forces. [0036] The actuation unit is a three-layer structure, with the top layer containing the external permanent magnet mounted on the robotic manipulator, the middle layer containing a cuboid workspace for holding patient and the localization unit, and the bottom layer containing the electromagnet array. [0037] The configuration of the external permanent magnet can be regular and Halbach array. [0038] The robotic manipulator can be an industrial robotic arm or a multiple-DOF mechanism. [0039] The electromagnet array contains 1 to 8 solenoidal electromagnets based on different application demands. [0040] Each of the spatial configuration of the electromagnet array can be adjusted based on different application demands. [0041] The magnetic object can be tethered or wireless. [0042] The internal permanent magnet configuration of the magnetic object can be regular and Halbach array. [0043] The localization unit comprises a printed circuit board and multiple three-axis Hall- effect sensors that formulate an array. [0044] The spatial configuration and plurality of the Hall-effect sensor array can be optionally adjusted based on different application demands. [0045] The configuration and plurality of the Hall-effect sensor array can be optionally adjusted. [0046] The localization algorithm further comprises the steps of: a. magnetic field ^_xyz{|} generated by the external permanent magnet of the magnetic object is measured by the the Hall-effect sensor array and further represented as a stacked matrix form ^_^ as ^^{00 ⋯ 0^ ~ ^~ Eqn (12);} b. modeled magnetic field in

as the stacked matrix form ^_^(^, ^_^^) which relative to the 5-DOF pose of the the magnetic object ^ and the position of ^^ Hall-effect sensor ^_^^. More specifically,^{^r(^, ^00) ⋯ ^r@^, ^0^D} ^_^@^, ^_^^D ≜ ^ ⋮ ⋱ ⋮ ^ Eqn (13); c. An

= _{^ − Eqn (14)} is constructed.5-DOF pose of the obtained by solving the the

optimization function. [0047] The stacked matrix ^_^ is measured from the chosen Hall-effect sensor layout which is sub-array of the Hall-effect sensor array with a specific plurality and distributed configuration. [0048] ^_^(^, ^_^^) could be represented by means of certain models including magnetic dipole model, magnetic multipole model, fitting model of magnetic field in mathematical paradigm. [0049] The optimization method for designing the configuration of the electromagnet array contains the steps of: a. the electromagnet array contains ^ electromagnets, with the pose of each the electromagnet represented by a vector ^_^(^, ^, ^) and its magnetic moment represented by a vector ^_^(^, ^). Both the number of the electromagnet array and orientation of each electromagnet is constrained by the size of the frame. b. a grid is generated to separate the cuboid workspace equally with ^ nodes. The position of each node is represented by a vector ^_^(^, ^, ^). c. the magnetic field on the specific node of the the grid ^_!(^, ^, ^) generated by an electromagnet ^_^(^, ^, ^) is represented by ^_^(^_^, ^_^) , with the superimposed magnetic field generated by the electromagnet array denoted as ^_^(^_^). d. a cost function is formulated to maximize the generated magnetic field ^_^(^_^) under the constraint of applied currents as ^ ^{= argmin} ^^{(^}^^{)‖ Eqn (1);} e. the norm of resultant

node of the grid is stored as ^, with its average and isotropy index represented as ^_{^ !} =^{^} " Eqn (2); Eqn (3);

f. the configuration of electromagnet array (i.e., number and electromagnet orientation) with the maximum ^_{^ !} and minimum ^_^#$ is selected as the optimal configuration. [0050] ^_^(^_^, ^_^) could be represented by means of certain models including magnetic dipole model, magnetic multipole model, fitting model of magnetic field in mathematical paradigm. [0051] The force-based method for designing the configuration of the external permanent magnet contains the steps of: a^{. the position of the external permanent magnet can be represented by a vector} ^_{6(^, ^, ^), with its magnetic moment represented by ^6.} b. the dimension of the internal permanent magnet is constrained by different application demands. The position of the internal permanent magnet of the magnetic object can be represented by a vector ^₇(^, ^, ^) , with its magnetic moment represented by ^₇. The position of the magnetic object is located below the external permanent magnet, with the height ℎ set as a constant (typically equal to the average abdominal height of human body). The orientation of the internal permanent magnet is set with its magnetic moment opposite to that of the external permanent magnet. M^{athematically, ^6(^, ^, ^) = ^7(^, ^, ^ + ℎ) Eqn (4);} ^_{6 = −^7 Eqn (5);} c. the magnetic force exerted on the magnetic object by the external permanent magnet is represented by a vector :(^₆ , ^₇, ^₆ , ^₇). Typically, the dragging and levitating force are emphasized, which can pull and levitate the magnetic object to perform desired tasks. d. by moving the internal permanent magnet along the axial direction of the external permanent magnet, the change of the magnetic force : is recorded. e. by comparing the dragging and levitating magnetic force, one can determine the configuration of the external permanent magnet (i.e., Halbach array or regular shape, dimension, and magnetization intensity). [0052] :(^₆ , ^₇, ^₆ , ^₇) could be represented by means of certain models including magnetic dipole model, magnetic multipole model, fitting model of magnetic force in mathematical paradigm. [0053] Using the magnetic dipole model, the magnetic field and force generated by a permanent magnet or an electromagnet can be expressed as ^_^ =^;<_@3^B_^^B_^⁾ − C_D^_^ =^;< ^_^^_^ Eqn (6);

:_^ =^F;< 5_=‖^>‖G H^^B_^⁾ + ^B_^^⁾ + (^B_^ ∙ ^)@^ − 5^B_^^B_^⁾DK ^_^ =^F;< 5₌ L_^^_^ Eqn (7); where ^_^ a magnetic

the displacement vector from the magnetic dipole to the desired position; ^^B_^is the normalized vector of ^_^ (i.e., ^_^^‖^_^^{‖^0}); ^ is a 3×3 identity matrix; ^ is the magnetic moment of the magnetic object to be controlled. [0054] Then by packing the magnetic field and force generated by the external permanent magnet in a matrix form, we have S^{^ ;} :^T P_"^{= <} 5_Y=Z^U_Z^{^ V ^} V_{Z[ WZ^Z}^X_Z^U_L\^{X ^}_P"^{= ](^}_P"^{)^}_P"^{Eqn (8);} [0055] The pose of the the joint angles of the

robot arm. By using the we b ^{^ de ^e} __{P"`</sub> =<sup>∏</sup><sub>^c0</sub> _<sub>^</sub><sup>^0</sup> = S<sup>P" P"</sup> f <sub>1</sub> T Eqn (9); where d<sub>P"`} involves all the

[0056] Similarly, by packing the magnetic field and force generated by the electromagnet array in a matrix form, we have ^ ^{;< ^ V ^0 ⋯ ^3 ^k 0 ⋯ f p0 ⋮ ⋱ ⋮} = ior Eqn (10); where ^k_^

current passing through its coil, p_^ represents the current applied to i-th electromagnet. [0057] Consequently, for a given magnetic field and force, we can calculate the currents applied to the electromagnetic coils and the pose of external permanent magnet by solving a formulated optimization problem as t ^ −^{^} −^{^} ;

g_{^ ≤ g ≤ g^ gw ^ ≤ gw ≤ gw ^} where ‖∙‖_t represents the Euclidean norm, (∙)^{^} and (∙)^{^} are the lower and upper bounds of the vector, respectively, ≤ is element-wise (general) inequality symbol. [0058] This invention provides a system and a method for generating magnetic fields to control magnetic objects. In one embodiment, said system and said method comprises an actuation unit, a localization unit, a design method, a localization algorithm, and a control algorithm, wherein A) said actuation unit is used for generating designed magnetic fields, which contains an electromagnet array, an external permanent magnet, a robotic manipulator, and a frame connecting and fixing these components. B) said magnetic object contains internal magnet(s) and responds to the magnetic fields generated by the actuation unit. C) said localization unit contains a Hall-effect sensor array for measuring magnetic fields generated by the said actuation unit and the said magnetic object. D) said localization algorithm for sensing the pose of the controlled object based on the magnetic fields measured by the said localization unit. E) said design method contains an optimization method for designing the configuration of the electromagnet array and a force-based method for designing the external permanent magnet. F) said control algorithm for planning the pose of the said external permanent magnet and the currents applied to the said electromagnet array to generate desired magnetic fields and forces. [0059] In one embodiment, said actuation unit is a three-layer structure, with the top layer containing said external permanent magnet mounted on said robotic manipulator, the middle layer containing a cuboid workspace for holding patient and said localization unit, and the bottom layer containing said electromagnet array. [0060] In one embodiment, the configuration of said external permanent magnet can be regular and Halbach array. [0061] In one embodiment, said robotic manipulator can be an industrial robotic arm or a multiple-DOF mechanism. [0062] In one embodiment, said electromagnet array contains 1 to 8 solenoidal electromagnets based on different application demands. [0063] In one embodiment, each of the spatial configuration of said electromagnet array can be adjusted based on different application demands. [0064] In one embodiment, said magnetic object can be tethered or wireless. [0065] In one embodiment, the internal permanent magnet configuration of said magnetic object can be regular and Halbach array. [0066] In one embodiment, said localization unit comprises a printed circuit board and multiple three-axis Hall-effect sensors that formulate an array. [0067] In one embodiment, the spatial configuration and plurality of said Hall-effect sensor array can be optionally adjusted based on different application demands. [0068] In one embodiment, configuration and plurality of said Hall-effect sensor array can be optionally adjusted. [0069] In one embodiment, said localization algorithm further comprises the steps of: a. magnetic field ^_$^^6^^ generated by said external permanent magnet of said magnetic object is measured by the said Hall-effect sensor array and further represented as a stacked matrix form ^_^ as ^^{00 0^} ^^{~ ⋯ ^~} ~ _{≜ ^} ⋮ ⋱ ⋮^{^ ^, ^ ∈ ℕ}_^^{Eqn (12);} b. modeled magnetic field as the stacked matrix

form ^_^(^, ^_^^) which pose said magnetic object ^ and said position of ^^ Hall-effect sensor ^_^^. More specifically, ^^{r(^, ^00) ⋯ ^r@^, ^0^D} ^_^^D ≜_{^ ⋮ ⋱ ⋮ ^} Eqn (13); c_{. An}

_{is constructed. 5-DOF pose} of said magnetic object ^ is then obtained by solving the said optimization function. [0070] In one embodiment, the stacked matrix ^_^ is measured from the chosen Hall-effect sensor layout which is sub-array of the Hall-effect sensor array with a specific plurality and distributed configuration. [0071] In one embodiment, ^_^(^, ^_^^) could be represented by means of certain models including magnetic dipole model, magnetic multipole model, fitting model of magnetic field in mathematical paradigm. [0072] In one embodiment, the optimization method for designing the configuration of the electromagnet array contains the steps of: a. said electromagnet array contains ^ electromagnets, with the pose of each said electromagnet represented by a vector ^_^(^, ^, ^) and its magnetic moment represented by a vector ^_^(^, ^). Both the number of said electromagnet array and orientation of each electromagnet is constrained by the size of said frame. b. a grid is generated to separate the cuboid workspace equally with ^ nodes. The position of each node is represented by a vector ^_^(^, ^, ^). c. the magnetic field on the specific node of the said grid ^_!(^, ^, ^) generated by an electromagnet ^_^(^, ^, ^) is represented by ^_^(^_^, ^_^) , with the superimposed magnetic field generated by the electromagnet array denoted as ^_^(^_^). d. a cost function is formulated to maximize the generated magnetic field ^_^(^_^) under the constraint of applied currents as

^ ^{= argmin ‖^}_^^{(^}_^^{)‖ Eqn (1);} ^ _{^ ^ ≤ ^ ≤ ^^} e. the norm of resultant magnetic field ‖^_^(^_^)‖ at each node of the grid is stored as ^, w^{ith its average and isotropy index represent} ^^{ed as} ^_{^ ! =" Eqn (2);} 0^{.3"^4.5} ^_^#$ =_%1 −^{()*(^)} +_,-.√"^01 Eqn (3); f. the configuration of with the maximum ^_{^ !}

[0073] In one embodiment, ^_^(^_^ , ^_^) could be represented by means of certain models including magnetic dipole model, magnetic multipole model, fitting model of magnetic field in mathematical paradigm. [0074] In one embodiment, the force-based method for designing the configuration of the external permanent magnet contains the steps of: g^{. the position of said external permanent magnet can be represented by a vector} ^_{6(^, ^, ^), with its magnetic moment represented by ^6.} h. the dimension of said internal permanent magnet is constrained by different application demands. The position of said internal permanent magnet of said magnetic object can be represented by a vector ^₇(^, ^, ^), with its magnetic moment represented by ^₇. The position of said magnetic object is located below said external permanent magnet, with the height ℎ set as a constant (typically equal to the average abdominal height of human body). The orientation of said internal permanent magnet is set with its m^{agnetic moment opposite to that of said external permanent magnet. Mathematically,} ^_{6(^, ^, ^) = ^7(^, ^, ^ + ℎ) Eqn (4); ^6 = −^7 Eqn (5);} i. the magnetic force exerted on said magnetic object by said external permanent magnet is represented by a vector :(^₆ , ^₇, ^₆ , ^₇). Typically, the dragging and levitating force are emphasized, which can pull and levitate said magnetic object to perform desired tasks. j. by moving said internal permanent magnet along the axial direction of said external permanent magnet, the change of the magnetic force : is recorded. k. by comparing the dragging and levitating magnetic force, one can determine the configuration of said external permanent magnet (i.e., Halbach array or regular shape, dimension, and magnetization intensity). [0075] In one embodiment, :(^₆ , ^₇ , ^₆ , ^₇) could be represented by means of certain models including magnetic dipole model, magnetic multipole model, fitting model of magnetic force in mathematical paradigm. [0076] In one embodiment, said control algorithm for planning the pose of the said external permanent magnet and the currents applied to the said electromagnet array to generate desired magnetic fields and forces contains the steps of: l. Using the magnetic dipole model, the magnetic field and force generated by a permanent magnet or an electromagnet can be expressed as ^_^ =^;< 5_=‖^>‖? @3^^B_^^^B_^⁾ − CD^_^ =^;< 5₌ ^_^^_^ Eqn (6); ;

a ^{magnetic moment of ^M; N4 is the magnetic permeability of free space; ^^ = ^O6# −} ^_{O^P$Q6 the}

_{vector from the magnetic dipole to the desired position; ^B^is} the normalized vector of ^_^ (i.e., ^_^^‖^_^^{‖^0} ); ^ is a 3×3 identity matrix; ^ is the magnetic moment of the magnetic object to be controlled. m. Then by packing the magnetic field and force generated by the external permanent magnet in a matrix form, we have S^{^ = ; ^ V U^ ^}_P"^{= ^}_P"^{Eqn (8);} n. The pose of the

the joint angles of the robot arm. By using the homogeneous transformation matrix, we have ^^{^0} S^{de ^e} = =^{P" P"} T Eqn (9); where d_P"` involves

o. Similarly, by packing the magnetic field and force generated by the electromagnet array in a matrix form, we have^{S^ ; ^ ⋯ ^ ^k 0 ⋯ f p0} :^{T = <} 5₌^{U ^ VX S}_L^{0 3 T j ⋮ ⋱ ⋮ n j ⋮ n} = ior Eqn (10); 6_{" YZZVZZZ WZ^Z[Z0ZZ ⋯ZZZ LZ3\ i YZfZZZ[ ⋯ZZZ ^kZ3\ qp3 o r} where ^k_^ represents the magnetic moment of i-th electromagnet with a unit current passing through its coil, p_^ represents the current applied to i-th electromagnet. p. Consequently, for a given magnetic field and force, we can calculate the currents applied to the electromagnetic coils and the pose of external permanent magnet by solving a formulated optimization problem as t m_r,i_gn sS^{^} :T − S^{^} :T − S^{^} :T s Eqn (11); O_{6# P" 6" t} s^{. t r^ ≤ r ≤ r^ rw ^ ≤ rw ≤ rw ^} g_{^ ≤ g ≤ g^ gw ^ ≤ gw ≤ gw ^} where ‖∙‖_t represents the Euclidean norm, r is the current vector applied to said electromagnet array with rw as its changing rate, g is the joint angle vector of said robotic manipulator with gw as its changing rate, (∙)^{^} and (∙)^{^} are the lower and upper bounds of the vector, respectively, ≤ is element-wise (general) inequality symbol.

Claims

What is claimed is: 1. A system for controlling a magnetic object in a working space, comprising: a. An actuation unit comprising an electromagnetic array, a robotic manipulator, and an external permanent magnet; b. A localization unit comprising a Hall-effect sensor array and a localization algorithm; and c. A computing unit comprising: i. a processor; ii. memory; and iii. program instructions, stored in the memory, that upon execution by the processor cause the computing unit to perform operations for designing a configuration of said electromagnetic array and/or a configuration of said external permanent magnet to generate a desired magnetic field to control said magnetic object. 2. The system of claim 1, wherein said actuation unit comprises a top layer, a middle layer and a bottom layer; wherein a. said top layer comprises said external permanent magnet mounted on said robotic manipulator; b. said middle layer comprises said working space and said localization unit; and c. said bottom layer comprises said electromagnet array. 3. The system of claim 1, wherein said operations comprise an optimization method for designing said configuration of the electromagnet array. 4. The system of claim 3, wherein said optimization method comprises the steps of: a. Representing pose and magnetic moment of each electromagnet in said electromagnetic array with a vector ^_^(^, ^, ^) and a vector ^_^(^, ^); and b. Generating a grid separating said working space equally with ^ nodes and representing position of each node by a vector ^_^(^, ^, ^); c. Representing magnetic field on a specific node generated by an electromagnet to be ^_^(^_^, ^_^) and a superimposed magnetic field generated by said electromagnet array to be ^_^(^_^); d^{. Formulating a cost function to maximize said superimposed magnetic field} ^_{^(^^) under constraint of applied current as: ^ = argmin ^ ‖^^(^^)‖;} e^{. Storing norm of resultant}

^{‖ at each node of said grid as} ^ _{and average and isotropy index represented as: ^ ^ ^ ! ="; ;}

f. Selecting a configuration of said electromagnet array with maximum ^_{^ !} and minimum ^_^#$. 5. The system of claim 1, wherein said operations comprise a force-based method for designing said configuration of the external permanent magnet. 6. The system of claim 5, wherein said force-based method comprises the steps of: a^{. Representing position of said external permanent magnets by a vector} ^_{6(^, ^, ^), with its magnetic moment represented by ^6;} b. Representing position of said magnetic object by a vector ^₇(^, ^, ^), with its magnetic moment represented by ^₇; c^{. Setting} ^_{6(^, ^, ^) = ^7(^, ^, ^ + ℎ); ^6 = −^7;} where h is a constant representing position of said magnetic object located below said external permanent magnet; d. Representing magnetic force exerted on said magnetic object by said external permanent magnet by a vector :(^₆ , ^₇ , ^₆ , ^₇); e. Recording change in magnetic force, : , by moving said magnetic object along axial direction of said external permanent magnet; and Determining a configuration for said external permanent magnet by comparing dragging and levitating magnetic force. 7. The system of claim 6, wherein said vector :(^₆ , ^₇ , ^₆ , ^₇) is represented by a model selected from a group consisting of magnetic dipole model, magnetic multipole model, fitting model of magnetic force in mathematical paradigm. 8. The system of claim 1, wherein said operations comprise a control algorithm for planning said configuration of the external permanent magnet and currents applied to said electromagnet array to generate desired magnetic fields. 9. The system of claim 8, wherein said control algorithm comprises the steps of: a. Expressing magnetic field and force generated by said external permanent m^{agnet or said electromagnet array using a magnetic dipole model:} ^_{^ =}^;< 5_{=‖^>‖? @3B^^^B^}⁾_{− CD^^ =}^;< 5_{= ^^^^;}

a space; ^_^ = ^_O6# − ^_O^P$Q6 the displacement vector from said magnetic d^{ipole to the desired position; ^B^is the normalized vector of ^ (i.e.,} ^_{‖ ^0}^{^} ^ _{^^‖ ); ^ is a 3×3 identity matrix; ^ is the magnetic moment of the} magnetic object to be controlled; b. packing magnetic field and force generated by said external permanent magnets in a matrix form:^{S^} :^T P_"^{= ;<} 5_Y=Z^U_Z^{^} V_Z[^V W_Z^Z^X_Z^{U^} L_\^{X ^}P"^{= ](^}P"^{)^}P"^; c. determining angles of said robotic

`<sup>b</sup> ^<sub>c0 ^</sub><sup>de ^e</sup> _<sub>P"</sub> =<sup>∏</sup> _<sup>^^0</sup> = S<sup>P" P"</sup> f <sub>1</sub> T; where d<sub>P"`</sub> involves all the joint angles g of the arm; d. packing magnetic field and force generated by

array in a matrix form S^{^ ;} :^{T = <} 5^{^ V ^0 ⋯ ^ ^k 0 3 ⋯ f p0} Y_=Z^U_{ZVZZZ WZ^}^X_Z^S_{[LZZ ⋯ L}^{T j ⋮ ⋱ ⋮ n j ⋮ n} = ior; 6_{" 0 ZZZZZ3\ i YZfZZZ[ ⋯ZZZ ^kZ\ 3 qp3 o r} where ^^k_^ represents the magnetic moment of i-th electromagnet with a unit current passing through its coil, p_^ represents the current applied to i-th electromagnet; e. Calculating currents applied to said electromagnet array and pose of said external permanent magnet by solving a formulated optimization problem: t m^{^} −^{^} − S^{^} s t where ‖∙‖_t (∙)^{^} are the lower and upper bounds of

wise (general) inequality symbol. 10. The system of claim 1, wherein said localization algorithm comprises the steps of: a. Measuring a magnetic field ^_xyz{|} generated by said magnetic object using said Hall-effect sensor array and representing said ^_xyz{|} in a stacked matrix form: ^^{00 0^ ~ ⋯ ^~} b. Expressing magnetic sensor in said Hall-

effect sensory array as a with respect to 5- DOF pose of said magnetic object, ^, and position of ^^ Hall-effect sensor ^_^^: ^^{r(^, ^00) ⋯ ^r(^, ^0^)} ^_; c. Constructing

^ ₌^argmin_{^^~ − ^^@^, ^^^D^; and}

d. Obtaining 5-DOF pose of the magnetic object, ^, by solving said optimization function. 11. The system of claim 10, wherein said stacked matrix form ^_^(^, ^_^^) comprises magnetic dipole model, magnetic multipole model, or fitting model of magnetic field in mathematical paradigm. 12. The system of claim 1, wherein said magnetic object comprises an internal permanent magnet configuration selected from regular or Halbach array. 13. The system of claim 1, wherein said external permanent magnet comprises a configuration selected from regular or Halbach array. 14. The system of claim 1, further comprising a system frame. 15. The system of claim 1, wherein said Hall-effect sensor array comprises one or more three-axis Hall-effect sensors.