DESCRI PTION
Title
SURGERY PLANNING SYSTEM, SURGICAL SYSTEM, AND SURGERY PLANNING METHOD
Technical Field
[0001 ] The present invention relates to a surgery planning system according to the preamble of independent claim 1 and more particularly to a surgery planning system that comprises a user interface (Ul). The present invention also relates to a surgical system and a surgery planning method.
[0002] Such a surgery planning system can be used for planning selective removal of hard tissue. The surgery planning system is particularly useful in association with a surgery system that comprises a laser beam generator for performing hard tissue ablation.
[0003] The term “hard tissue” as used in the context of the invention can relate to a tissue which is mineralized and/or has a firm intercellular matrix. The hard tissue can particularly be bone tissue and/or cartilage tissue or similar.
Background Art
[0004] US 8 109 956 B2 discloses techniques of tissue removal. These techniques rely on surgeon experience and do not provide adequate guidance and support to the surgeon.
[0005] US 2013/0317363 A1 , which serves as basis for the preamble of claim 1 , discloses an ablation planning and navigation system. The planning system includes a memory configured to store a plurality of medical images and a controller configured to render the plurality of images in three dimensions, automatically segment the plurality of images to demarcate a target area, and automatically determine a treatment plan based on the target area. The navigation system includes an ultrasound device having a fiducial pattern.
[0006] Such a planning system assists a surgeon in the complex task of identifying a suitable ablation procedure but still has various shortcomings. For illustration, it is still challenging to assess and compare the suitability of various paths for performing hard tissue ablation in accordance with an objective criterion.
[0007] Therefore, there is a need for an improved surgery planning system and surgery planning method that provides guidance and support in surgery planning. There is in particular a need for techniques which assist a surgeon in assessing the suitability of various access points when evaluated in accordance with an objective criterion.
Disclosure of the Invention
[0008] According to the invention this need is settled by a surgery planning system as it is defined by the features of independent claim 1 , and by a surgery planning method as it is defined by the features of independent claim 14. Preferred embodiments are subject of the dependent claims.
[0009] In one aspect, the invention is a surgery planning system, comprising a user interface and at least one integrated circuit configured to process medical image data of a subject and control the user interface to output a graphical representation of the medical image data. The user interface is configured to enable input specifying a target and one or several prohibited regions relative to a graphical representation of the medical image data. The at least one integrated circuit is configured to use the processed image data, the target, and the one or several prohibited regions to identify an acceptable trajectory and to identify at least one additional trajectory to the target. The acceptable trajectory passes through hard tissue and has a distance equal to or greater than a safety margin distance from each prohibited region of the one or several prohibited regions, such as soft tissue of the nerval or vascular system. The at least one integrated circuit is configured to control the user interface to output the acceptable trajectory and the at least one additional trajectory in a visually distinguishable manner. [0010] The surgery planning system assesses several possible trajectories and outputs information on which of the trajectories is acceptable in the sense that it bypasses all prohibited regions by at least the safety margin distance. Suitable paths for performing hard tissue ablation and, in particular, suitable entry and target points can thereby be identified in an intuitive manner. The association of a trajectory and its suitability for performing hard tissue ablation is immediately apparent from the visually distinguishable representations. Safety is further enhanced by providing a user interface that allows prohibited regions to be defined. The prohibited regions may include nerve and/or vascular tissue or cavities. Safety is further enhanced by providing a safety margin distance around any of the prohibited regions, to prevent an ablation modality (e.g., a laser beam) from passing closer than the safety margin distance along any of the prohibited regions. The surgery planning system may allow the safety margin distance to be set or adjusted by a user, thereby providing enhanced controllability.
[0011 ] The hard tissue may be tissue which is mineralized and/or has a firm intercellular matrix. The hard tissue may be bone tissue and/or cartilage tissue or similar.
[0012] Preferably, the at least one additional trajectory comprises an additional trajectory which passes through at least one of the prohibited regions and/or an additional trajectory which is within the safety margin distance from at least one of the prohibited regions.
[0013] The at least one additional trajectory can be determined in order to visualize the effect of a variation in entry point on the pathways to the target (which may be or may comprise a target point). For illustration, the surgery planning system can allow the user to test out various candidate entry points on the hard tissue. The resultant at least one additional trajectory is displayed, concurrently or sequentially with the acceptable trajectory, to thereby indicate how differences in entry point affect the arrangement of the trajectory with respect to the prohibited regions.
[0014] By displaying this additional trajectory in a manner which allows it to be visually distinguished from the acceptable trajectory, the effect of trajectory arrangement relative to the subject’s anatomy on the suitability of the trajectory can be conveyed to the user in an efficient manner. [0015] Preferably, the one or several prohibited regions are regions which contain delicate tissue, in particular soft tissue. The one or several prohibited regions may be regions in which soft tissue of the nerval or vascular system are disposed.
[0016] Preferably, the at least one additional trajectory comprises a first additional trajectory that passes through at least one of the prohibited regions and/or a second additional trajectory that does not pass through any of the prohibited regions but which does not keep the safety margin distance from at least one of the prohibited regions. By displaying the first additional trajectory and/or the second additional trajectory in addition to the acceptable trajectory, the effect of trajectory arrangement relative to the subject’s anatomy on the suitability of the trajectory can be conveyed to the user in an efficient manner.
[0017] Preferably, the at least one additional trajectory comprises a first additional trajectory and a second additional trajectory, wherein the first additional trajectory passes through a prohibited region of the one or several prohibited regions, wherein the second additional trajectory does not pass through any prohibited region of the one or several prohibited regions, and wherein the at least one integrated circuit is configured to control the user interface to output the acceptable trajectory, the first additional trajectory, and the second additional trajectory in the visually distinguishable manner.
[0018] By displaying the acceptable trajectory, the first additional trajectory and the second additional trajectory in a manner which allows them to be visually distinguished from each other, the effect of trajectory arrangement relative to the subject’s anatomy on the suitability of the trajectory can be conveyed to the user in an efficient manner.
[0019] The visually distinguishable manner of displaying the various trajectories may comprise any one or any combination of: using different colors, using different lines widths, using different broken line types (e.g., solid, dotted, dashed, dashed-dotted), arranging labels or symbols on the various trajectories, so that the different types of trajectories can be visually distinguished.
[0020] Preferably, the user interface is configured to enable input specifying several entry points for a trajectory.
[0021 ] Thereby, the surgery planning system provides an input modality that allows the user to test out various entry points for a trajectory. [0022] Preferably, the at least one integrated circuit is configured to identify, for each of the trajectory’s entry points, whether a trajectory for hard tissue ablation from the entry point to the target point is the acceptable trajectory.
[0023] This allows the user to test various trajectory entry points to determine which entry point(s) result in an acceptable trajectory to safely reach the target point through hard tissue. Each of the trajectories for hard tissue ablation may be a straight trajectory suitable for, e.g., laser beam ablation of the hard tissue. The surgery planning system provides feedback on the suitability of the resultant trajectory.
[0024] In other words, the surgery planning system can, for an entry point for a trajectory that may be set by the user or tested automatically by the surgery planning system, determine how a trajectory from that entry point to the target (e.g., a target point) is arranged relative to the one or several prohibited regions. The result of the determination may be graphically output. For illustration, the surgery planning system may generate and provide graphical output which indicates whether there is a trajectory from the entry point to the target which bypasses all prohibited regions by at least the safety margin distance.
[0025] Preferably, the user interface is configured to enable displacement of an interface element relative to the graphical representation to specify the several entry points, wherein the user interface is configured to sequentially display trajectories from the several entry points to the target point in a manner which depends on whether the trajectory is the acceptable trajectory.
[0026] This allows the user to test various entry points by moving around the interface element, e.g., in a drag-and-drop technique. The surgery planning system provides feedback on the suitability of the resultant trajectory.
[0027] Preferably, the user interface is configured to enable displacement of the interface element in three dimensions.
[0028] This configuration enables and facilitates procedures in which the interface element is displaced along the 3D shape of a skull surface or other anatomical structure, to thereby determine a suitable trajectory (and, thus, entry point on the hard tissue surface) for hard tissue ablation. [0029] Preferably, the user interface is configured to enable input setting or adjusting the safety margin distance, wherein the at least one integrated circuit is configured to use the set or adjusted safety margin distance to identify the acceptable trajectory and the at least one additional trajectory.
[0030] Enhanced configurability and control of the surgery planning method is attained thereby.
[0031 ] Preferably, the user interface is configured to enable input specifying a diameter of a hard tissue ablation volume, wherein the at least one integrated circuit is configured to use the specified diameter to determine that every point of the hard tissue volume along the acceptable trajectory has a distance equal to or greater than the safety margin distance from each prohibited region of the one or several prohibited regions.
[0032] Enhanced configurability and control of the surgery planning method is attained thereby, taking into consideration the diameter of the tissue ablation volume. The diameter may be measured transverse to the respective trajectory and may be variable or constant along the trajectory. The diameter may be determined by the diameter of a tool or other elongate object that is to be passed through the cavity formed by hard tissue ablation.
[0033] Preferably, the at least one integrated circuit is configured to use the processed image data to assist the user in specifying the prohibited regions. The at least one integrated circuit may determine boundaries of a hard tissue lumen that contains nerve or vascular tissue. The at least one integrated circuit may control the user interface to enable a user input specifying a volume through which the laser beam is not allowed to pass and may automatically enlarge that volume to extend to boundaries of hard tissues. The surgery planning system may thereby provide a function that can be thought of a “lasso” function that captures all of a 3D volume prohibited for the trajectory, in response to an input at the user interface that specifies part of the 3D volume as containing tissue that must not be ablated.
[0034] Preferably, the user interface is configured to enable input specifying the one or several prohibited regions as volumetric regions relative to the graphical representation. [0035] Thereby, the surgery planning system provides assistance and support in the highly complex and non-intuitive task of determining suitable trajectories relative to a 3D bone structure when the prohibited regions are 3D volumes.
[0036] Preferably, the acceptable trajectory is a laser beam trajectory for performing laser beam ablation. The surgery planning system according to the present invention is particularly suitable for use with laser beam ablation systems.
[0037] Preferably, the medical image data represents a volume containing a lateral skull base of the subject and the acceptable trajectory is a laser beam trajectory for laser ablation of hard tissue in the lateral skull base. The surgery planning system according to the present invention is particularly suitable for planning lateral skull base tissue ablation interventions.
[0038] Preferably, the surgery planning system comprises an output interface to output one or several of the acceptable trajectory and/or control data and/or control signals for controlling components of a surgical system. Thereby, the results of the processing performed by the surgery planning system may be output for use in an automatic control of components of a surgical system.
[0039] Preferably, the medical image data comprises a computer tomography (CT) image and/or a magnetic resonance imaging (MRI) image. Other medical imaging techniques may be used.
[0040] Thereby, the surgery planning system is enabled to determine geometrical features of the subject’s anatomy.
[0041 ] According to another aspect, the invention is a surgical system, which comprises a surgical tool which comprises a laser beam generator, a robot arm on which the laser beam generator is arranged, and the surgery planning system according to an aspect or embodiment. The surgical system is configured to use the acceptable trajectory to control the robot arm and the laser beam generator.
[0042] Thereby, the results of the surgery planning system may be used for the automatic control of components of the surgery system, providing enhanced accuracy and safety. [0043] Preferably, the surgical system is a system for performing hard tissue ablation in a lateral skull base of the subject for removal or treatment of cancerous or diseased tissue such as tumors.
[0044] The surgery planning system and method disclosed herein are particularly suitable for hard tissue removal in the lateral skull base, where conventional surgery techniques are particularly difficult to implement because of the proximity to delicate and/or critical tissue such as nerval or vascular structures.
[0045] The surgery planning system may comprise a control system which may be coupled to or integral with the surgery planning system. The control system may be configured to control the robot arm and laser beam generator to perform laser ablation in the lateral skull base of the subject for removal or treatment of cancerous or diseased tissue such as tumors.
[0046] Thereby, the surgery planning system and method provide assistance in the complex process of hard tissue removal in the lateral skull base, where conventional surgery techniques are particularly difficult to implement because of the proximity to delicate and/or critical tissue.
[0047] The surgical system may be an automatic surgical system or a surgery assist system.
[0048] The surgical system may further comprise an imaging modality configured to capture the medical image data and provide the medical image data to the surgery planning system.
[0049] The surgical system may further comprise components for determining a relative position of the laser beam generator relative to the subject’s skull, such as a 3D positioning system. The implementation of such a positioning system (e.g., by means of retroreflectors, light sources, and at least two cameras) is known to the skilled person.
[0050] The robot arm may have at least five degrees of freedom. This allows both the location of a laser beam output orifice (three degrees of freedom) and an orientation of the laser beam axis (two degrees of freedom) to be automatically adjusted under the control of the control system. [0051 ] According to a further aspect, the invention is a surgery planning method performed by a surgery planning system, wherein the surgery planning system comprises a user interface and at least one integrated circuit, the surgery planning method comprising: processing, by the at least one integrated circuit, image data of a subject, controlling, by the at least one integrated circuit, the user interface to output a graphical representation of the medical image data, enabling, by the user interface, input specifying a target and one or several prohibited regions relative to a graphical representation of the medical image data, using, by the at least one integrated circuit, the processed image data, the target, and the one or several prohibited regions to identify an acceptable trajectory and at least one additional trajectory to the target, herein the acceptable trajectory passes through hard tissue and has a distance equal to or greater than a safety margin distance from each prohibited region of the one or several prohibited regions, and controlling, the at least one integrated circuit, the user interface to output the acceptable trajectory and the at least one additional trajectory in a visually distinguishable manner.
[0052] The surgery planning method assesses several possible trajectories and outputs information on which of the trajectories is acceptable in the sense that it bypasses all prohibited regions by at least the safety margin distance. Suitable paths for performing hard tissue ablation and, in particular, suitable access points can thereby be identified in an intuitive manner. The association of a trajectory and its suitability for performing hard tissue ablation is immediately apparent from the visually distinguishable representations. Safety is further enhanced by providing a user interface that allows prohibited regions to be defined. The prohibited regions may include nerve and/or vascular tissue. Safety is further enhanced by providing a safety margin distance around any of the prohibited regions, to prevent an ablation modality (e.g., a laser beam) from passing closer than the safety margin distance along any of the prohibited regions. The surgery planning system may allow the safety margin distance to be set or adjusted by a user, thereby providing enhanced controllability.
[0053] The surgery planning method can be performed by the surgery planning system and/or surgical system according to an aspect or embodiment of the invention.
[0054] Preferably, the medical image data represents a volume containing a lateral skull base of the subject, wherein the acceptable trajectory is a trajectory for performing laser ablation of hard tissue in the lateral skull base for removal or treatment of cancerous or diseased tissue such as tumors. The surgery planning system lends itself to being used for lateral skull base tissue ablation, where conventional assistance techniques are difficult to implement due to the proximity of delicate tissue.
[0055] Additional features of the surgery planning method and the effects attained thereby correspond to the features described in association of the surgery planning system.
[0056] According to another aspect of the invention, there is provided a control method for a surgical system which comprises using the determined acceptable trajectory for controlling a robot arm and laser beam generator of the surgical system.
[0057] The control method can provide the control signals required to control the robot arm and laser beam generator to remove hard tissue in a lateral skull base portion by way of laser ablation. The position and operation of the laser beam generator is controlled by the control system, using results of an image evaluation. High accuracy and precision in hard tissue ablation is attained thereby. The surgical system does not exert mechanical stress on hard tissue of the lateral skull base that is not ablated, allowing the adverse effects of electromechanical tools to be avoided. By positioning and controlling the laser beam generator such that, upon activation of the laser source, the laser beam impinges onto the lateral skull base, the control method is suitable for specific procedures such forming an access through the lateral skull base for treatment or removal of cancerous or diseased tissue such as tumors.
[0058] Additional features of the control method and the effects attained thereby correspond to the features described in association of the surgical system.
[0059] In another aspect, the invention is a method of removing hard tissue (e.g., bone tissue) of a lateral skull base of a subject, the method comprising: receiving, by a control system, image data that represents a volume containing at least part of a skull base of a subject; evaluating, by the control system, the medical image data to identify an acceptable laser trajectory for laser ablation of hard tissue in the skull base; using, by the control system, the evaluated image data to control a laser beam generator to cause the laser beam generator to emit a laser beam that impinges on the lateral skull base.
[0060] The method is configured in such a manner that it can remove hard tissue in a lateral skull base by way of laser ablation. The position and operation of the laser beam generator are controlled by the control system, using results of an image evaluation. High accuracy and precision in hard tissue ablation is attained thereby. The surgical method does not exert mechanical stress on hard tissue of the lateral skull base that is not ablated, allowing the adverse effects of electromechanical tools to be avoided. By emitting the laser beam onto the lateral skull base, the surgical method is configured for use in specific procedures such as treatment and/or removal of cancerous or diseased tissue such as tumors by forming an access in the lateral skull base.
[0061 ] The method may further comprise using, by the control system, the evaluated image data to control a robot arm on which the laser beam generator is arranged. A desired position and orientation of the laser beam generator can be controlled in an accurate and precise manner, using the evaluated image data.
[0062] Determining the acceptable trajectory may comprise performing the surgery planning method according to an aspect or embodiment of the invention.
[0063] According to another aspect of the invention, there is disclosed machine- readable instruction code which, when executed by one or several programmable circuits (e.g., of the control system and/or surgery planning system), cause execution of the method according to an aspect or embodiment of the invention.
[0064] According to another aspect of the invention, there is disclosed a non-transitory storage medium storing machine-readable instruction code which, when executed by one or several programmable circuits (e.g., of the control system and/or surgery planning system), cause execution of the method according to an aspect or embodiment of the invention.
Brief Description of the Drawings
[0065] The surgery planning system, surgical system and surgery planning method according to the invention are described in more detail hereinbelow by way of an exemplary embodiment and with reference to the attached drawings, in which:
Fig. 1 shows a block diagram of a surgery planning system according to an embodiment of the invention;
Fig. 2 shows a surgical system comprising the surgery planning system of Fig. 1 ;
Fig. 3 shows a user interface of the surgery planning system of Fig. 1 ;
Fig. 4 shows another user interface of the surgery planning system of Fig. 1 ;
Fig. 5 shows the user interface of Fig. 4 after displacement of an interface element; Fig. 6 shows the user interface of Fig. 4 after further displacement of an interface element;
Fig. 7 shows another user interface of the surgery planning system of Fig. 1 ;
Fig. 8 shows the user interface of Fig. 7 after displacement of an interface element;
Fig. 9 shows the user interface of Fig. 7 after further displacement of an interface element;
Fig. 10 shows a flow chart of a method according to an embodiment of the invention.
Fig. 11 shows a trajectory determined using a surgery planning system or method according to another embodiment of the invention.
Fig. 12 shows trajectories determined using a surgery planning system or method according to another embodiment of the invention.
Description of Embodiments
[0066] In the following description certain terms are used for reasons of convenience and are not intended to limit the invention. The terms “right”, “left”, “up”, “down”, “under" and “above" refer to directions in the figures. The terminology comprises the explicitly mentioned terms as well as their derivations and terms with a similar meaning. Also, spatially relative terms, such as "beneath", "below", "lower", "above", "upper", "proximal", "distal", and the like, may be used to describe one element's or feature's relationship to another element or feature as illustrated in the figures. These spatially relative terms are intended to encompass different positions and orientations of the devices in use or operation in addition to the position and orientation shown in the figures. For example, if a device in the figures is turned over, elements described as "below" or "beneath" other elements or features would then be "above" or "over" the other elements or features. Thus, the exemplary term "below" can encompass both positions and orientations of above and below. The devices may be otherwise oriented (rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein interpreted accordingly. Likewise, descriptions of movement along and around various axes include various special device positions and orientations.
[0067] To avoid repetition in the figures and the descriptions of the various aspects and illustrative embodiments, it should be understood that many features are common to many aspects and embodiments. Omission of an aspect from a description or figure does not imply that the aspect is missing from embodiments that incorporate that aspect. Instead, the aspect may have been omitted for clarity and to avoid prolix description. In this context, the following applies to the rest of this description: If, in order to clarify the drawings, a figure contains reference signs which are not explained in the directly associated part of the description, then it is referred to previous or following description sections. Further, for reason of lucidity, if in a drawing not all features of a part are provided with reference signs it is referred to other drawings showing the same part. Like numbers in two or more figures represent the same or similar elements.
[0068] The techniques disclosed herein may be used for planning a surgery that can be performed using a laser beam generator, without being limited thereto.
[0069] As used herein, the term “laser beam generator” encompasses at least one component that outputs a laser beam. The laser beam generator may be formed by an assembly of optical components that outputs the laser beam. The laser beam generator may but does not need to comprise a laser. The optical components from which the laser beam generator is formed may be or may comprise an end of an optical fiber or other lightguide or lenses, shutters, or other optical components. These optical components may be mounted on a robot arm so as to be moveable. These optical components may be separate from the laser. The laser itself may be stationary and need not be displaceable jointly with the optical components that act as laser beam generator.
[0070] Fig. 1 shows a block diagram of a surgery planning system 20. The surgery planning system 20 is configured to perform various processing functions disclosed herein for determining and visualizing the acceptability of a trajectory for hard tissue removal. The surgery planning system 20 may execute machine-readable instruction code stored in a storage system 23 to perform the various processing and control functions disclosed herein. The surgery planning system 20 may be implemented as a computer or server communicatively coupled, via wired or wireless connections, to components of a surgical system (such as a robot arm 11 and laser beam generator 12 which will be disclosed in more detail below).
[0071 ] The surgery planning system 20 is configured to (i) process medical image data (e.g., by performing image segmentation) that represents a 3D volume of a subject, (ii) control a user interface (III) 22 to receive user input specifying a target and one or several prohibited regions through which a laser beam or other tool for hard tissue removal must not pass, and (iii) use the processed image data to identify and visualize on the III 22 the acceptability of a trajectory for the laser beam or other hard tissue removal tool.
[0072] The surgery planning system 20 is configured to use an objective criterion, namely distance from the prohibited region(s), for assessing acceptability. The surgery planning system 20 may classify trajectories (which may be straight lines terminating at the specified target). Various classes may be used, such as
- a class for acceptable trajectories which have a distance that is at least equal to a safety margin distance from each of the prohibited region(s), and
- a class of non-acceptable trajectories which pass through or otherwise do not have a distance that is at least equal to a safety margin distance from each of the prohibited region(s).
[0073] As conventional for 3D objects, the “distance” of the trajectory from a prohibited region is determined as minimum of any point on the trajectory from any point of the prohibited region, with the minimum being determined as minimization over all points on the trajectory and all points of the prohibited region.
[0074] The surgery planning system 20 is configured to output trajectories differently, depending on whether they maintain at least the safety margin distance from each of the prohibited region(s). The surgery planning system 20 may provide further differentiation in visualization, e.g., by rating different acceptable trajectories in accordance with how far they are spaced from the prohibited region(s) and/or by rating non-acceptable trajectories in accordance with whether they pass or do not pass through any of the prohibited regions.
[0075] The assessment is performed based on objective criteria, using image processing (e.g., image segmentation) of the medical image data.
[0076] The surgery planning system 20 comprises one or several interface(s) 21. The one or several interface(s) 21 may comprise a digital interface configured to receive the medical image data 41 . The medical image data 41 may be volumetric data comprising a plurality of voxels. The one or several interface(s) 21 may comprise a digital and/or analog interface to output control data or signals 42 for controlling components of a surgical system (such as the robot arm 11 and the laser beam generator 12 described in association with Fig. 1 ).
[0077] The surgery planning system 20 comprises the III 22. A processing system 30 of the surgery planning system 20 controls the III 22 to enable user input that is relevant to assessing the acceptability of trajectories and thereby identifying at least one acceptable trajectory.
[0078] The III 22 is an input-output interface. The surgery planning system 20 may control the III 22 to output a graphical representation of the medical image data and enable user input in relation to the output graphical representation of the medical image data. The graphical representation may include a graphical representation of the skull base. The III 22 enables a user input that specifies a target relative to a graphical representation of the medical image data. The III 22 enables a user input that specifies prohibited regions through which laser light must not pass, in relation to the graphical representation. The processing system 30 may assist in the definition of the prohibited regions, in particular by using results of an image segmentation to determine a 3D volume through which the laser beam must not pass and which contains pixels or voxels indicated by the user as being prohibited for the laser beam. The III 22 may enable a user input that specifies a safety margin distance by which laser light is to bypass any of the prohibited regions. The safety margin distance may but does not need to be input relative to the graphical representation. The III 22 may enable additional input operations, such as inputting a desired diameter or radius of a passage formed in the skull base by laser ablation.
[0079] Generally, as used herein, the term “safety margin distance” refers to a non-zero distance.
[0080] The processing system 30 may comprise one or several integrated circuit(s) (IC(s)) 35. The one or several IC(s) 35 may comprise one or several application specific integrated circuit(s) (ASIC(s)), controller(s), processor(s), field programmable gate array(s) (FPGA(s)), quantum gates, or combinations thereof. The processing system 30 may execute machine-readable instruction code to perform any of the functions disclosed herein. [0081 ] The processing system 30 is configured to perform an image processing 31 to process the medical image data 41. The image evaluation 31 may comprise an image segmentation 32. The image segmentation 32 may determine a 3D contour of an anatomical structure (e.g., a skull base access through the mastoid portion). The processing system 30 may use the 3D contour for various purposes, such as ensuring that no tissue ablation is performed outside of hard tissue.
[0082] The image evaluation 31 may perform functions in addition to image segmentation. For illustration, geometrical characteristics of the anatomical structure may be determined (such as a cloud of points on the surface; dimensions of various parts of the skull base; etc.) and used for assessing the viability of various laser trajectories through the anatomical structure.
[0083] The processing system 30 is configured to perform a trajectory identification 33. The trajectory identification 33 is configured to identify one or several acceptable trajectories that bypass all prohibited regions by at least the safety margin distance. The trajectory identification 33 is configured to also identify non-acceptable trajectories.
[0084] The processing system 30 is configured to perform a III control 34. The III control 34 is configured to generate control signals or data for visualizing acceptable trajectories and non-acceptable trajectories on the III 22 relative to a graphical representation of the medical image data.
[0085] In addition to visualizing acceptable and non-acceptable trajectories on the III 22, the surgery planning system 20 may be configured to use the identified acceptable trajectory to generate and output the control data or signals 42.
[0086] Fig. 2 shows a surgical system 1 according to an embodiment of the invention. The surgical system 1 comprises the surgery planning system 20. While the surgery planning system 20 may be integrated into a control system 2 of the surgical system, the surgery planning system 20 may be a separate component communicatively coupled to the control system 2.
[0087] The surgical system 1 comprises a robot arm 11. The surgical system 1 comprises a surgical tool comprising a laser beam generator 12. The laser beam generator 12 is arranged on the robot arm 11. As noted above, the laser beam generator 12 may be formed by optical components (such as an end of an optical fiber or other lightguide, an output optics, or similar) which may be separate from the laser. The laser itself need to be positioned on the robot arm. The laser beam generator 12 configured to emit a laser beam 10 directed onto a skull base to perform tissue ablation of hard tissue of the skull base. The surgical system 11 comprises the control system 2 configured to control the robot arm 11 and the laser beam generator 12. The control system 2 is configured to evaluate image data received from a source of image data 18 and use the evaluated image data to control the robot arm 11 and the laser beam generator 12 such that laser ablation is performed for hard tissue of, e.g., a skull base of a subject’s skull 19.
[0088] The laser beam generator 12 of the surgical system according to the invention may have a configuration and operation as disclosed in EP 3897438 A1 and/or EP 4 013 329 A1 .
[0089] Processing of the medical image data performed by the surgery planning system 20 may comprise a segmentation to identify surfaces of an anatomical structure, e.g., of the skull base of the subject’s skull 19. The control system 2 may use the processed image data (e.g., the segmented image data) to control the position, orientation, and output of the laser beam generator 12. The subject may be supported on a support 13 during the laser ablation disclosed herein.
[0090] The control system 2 may use the processed medical image data in various ways. As described above, the surgery planning system 20 of the control system 20 uses the processed image data to identify acceptable trajectories for tissue ablation. The control system 2 may additionally use the processed image data to determine whether laser ablation of hard tissue of the skull base is to be continued.
[0091 ] As described above, the control system 2 (and more particularly the surgery planning system 20) uses the processed image data to enable user input specifying constraints or other conditions for laser ablation of hard tissue of the skull base. The surgery planning system 20 comprises the III 22 to output a graphical representation of the medical image data. The III 22 is configured to enable user input specifying the target relative to a graphical representation of the medical image data, prohibited regions through which laser light or another tissue removal tool must not pass, a safety margin distance by which laser light is to bypass any of the prohibited regions. The III 22 may be configured to enable user input specifying various candidate entry points for the laser beam and visualize whether and to what degree the respective candidate entry points are suitable when considering the target, the prohibited regions, and the safety margin distance.
[0092] The control system 2 (and more particularly the surgery planning system 20) may use the processed image data to provide assistance in the definition of, e.g., the prohibited regions. The control system 2 may determine boundaries of a hard tissue lumen that contains nerve and/or vascular tissue (especially the facial nerve) and on which the laser beam must not impinge. The control system 2 may control the III to enable a user input specifying a volume through which the laser beam is not allowed to pass and may automatically enlarge that volume to extend to boundaries of hard tissues. For illustration, when the III enables the user to specify a prohibited region through which a nerve or a blood vessel passes or which contains brain tissue, the control system 2 may automatically extend the prohibited region to a three-dimensional volume delimited by hard tissue boundaries, which in turn may be determined automatically by the processed image data (e.g., as results of image processing). The surgery planning system 20 may thereby provide a function that can be thought of a “lasso” function that captures all of a 3D volume prohibited for the laser beam, in response to an input at the III that specifies part of the 3D volume as containing tissue that must not be acted upon by the laser beam or another hard tissue removal tool.
[0093] The source of image data 18 may comprise a data storage system that has stored therein medical image data. The medical image data may comprise a computer tomography (CT) and/or magnetic resonance imaging (MRI) image. Alternatively, or additionally, the source of image data 18 may comprise an image acquisition device that captures CT and/or MRI image data. The source of image data 18 may be or may comprise a CT system and/or an MRI system.
[0094] The control system 2 is configured in such a way that it controls the robot arm 11 and the laser beam generator 12 to perform laser ablation of hard tissue of the skull base. The control system 2 may be configured to perform any one or any combination of laser ablation procedures, which may include any one, several, or all of: hard tissue ablation for forming an access in a skull base (e.g., for tumor treatment or removal), an antrostomy, a mastoidectomy, mastoidectomy templating, a tympanotomy, other one-, two-, or three- dimensional ablations of hard tissue along one or several delicate structures. By laserablating the hard tissues, healing of the hard tissue remnant after the ablation is improved. In particular, the surfaces of the hard tissue remaining after the laser ablation is hard tissue that has been exposed to laser ablation but which has not been exposed to the mechanical stresses exerted by an electromechanical tool.
[0095] The control system 2 may be configured to use the output of a navigation system to control the robot arm 11 and/or laser beam generator 12. The navigation system may include optical elements including light sources 17, reflectors 16, and cameras 14’, which may be arranged on the laser beam generator 12, a reference 15 mounted on the subject’s head 19, and a navigation system component 14. It will be appreciated that light source(s), reflector(s), and camera(s) may be distributed in various ways among the laser beam generator 12, reference 15, and navigation system component 14 to allow determination of the relative position and orientation of the laser beam generator 12 and subject’s head 19. Such navigation systems are available to the skilled person and their operation is understood by the skilled person. The relative position and orientation as provided by the navigation system may be used by the control system 2 to determine which further adjustments need to be made to ensure that the laser beam 10 impinges on the anatomical structure, e.g., the skull base, in a desired manner.
[0096] Fig. 3 is a schematic representation of graphics output via the III 22. The surgery planning system 20 is configured to control the III 22 to output a graphical representation 100 of an anatomical structure (such as the skull base). The surgery planning system 20 is configured to control the III 22 to enable a user to input a target 95. The surgery planning system 20 is configured to control the III 22 to enable a user to specify prohibited regions 93, 94 through which the laser beam or other hard tissue removal tool must not pass.
[0097] The surgery planning system 20 may control the III 22 such that assistance is provided in these tasks. For illustration, the surgery planning system 20 may use the results of the image processing to determine the boundary or boundaries of one or several prohibited region(s) 94. At least one of the prohibited region(s) 94 may have a boundary coinciding with part of a 3D surface contour of the anatomical structure (e.g., of the skull base). The user needs to specify only part of a lumen containing tissue that must not be treated by the laser beam, and the surgery planning system 20 may be configured to then automatically determine the 3D volume that contains what has been specified by the user (which may be a point or a smaller volume) and extends to the 3D surface of the anatomical structure (e.g., of the skull base). [0098] The surgery planning system 20 may be configured to control the III 22 such that the user is enabled to input values for adjustable parameters relevant to assessing the acceptability of a trajectory. For illustration, the III 22 may provide an input element 91 enabling the user to specify the safety margin distance 96. The safety margin distance specifies the minimum spacing that is to be maintained between the trajectory and any one of the prohibited region(s) 93, 94. Additionally or alternatively, the III 22 may provide another input element 92 enabling the user to specify a diameter of a volume of hard tissue that is to be removed along the trajectory.
[0099] The surgery planning system 20 may use the results of the image processing (e.g., a 3D surface contour of the skull base), the target point, and the prohibited zones 93, 94 to automatically determine whether a trajectory 102 to the target point 95 passes through hard tissue such that it is spaced by at least the safety margin distance 96 from all of the prohibited regions 93, 94. At least one of the prohibited regions 93, 94 may be a lumen through which a nerve and/or blood vessel passes or which contains brain tissue.
[00100] The surgery planning system 20 may use the results of the image processing (e.g., a 3D surface contour of the skull base) and the prohibited zones 93, 94 to automatically determine whether a trajectory 104 to the target 95 passes through the hard tissue such that it does not intersect any of the prohibited regions 93, 94 but is spaced by less than the safety margin distance 96 from at least one of the prohibited regions 93, 94.
[00101 ] The surgery planning system 20 may use the results of the image processing (e.g., a 3D surface contour of the skull base) and the prohibited zones 93, 94 to automatically determine whether a trajectory 106 to the target 95 passes through the hard tissue and intersects at least one of the prohibited regions 93, 94.
[00102] The various different trajectories 102, 104, 106 may be determined in response to a user input enabled by the III 22, which user input specifies an entry point 101 , 103, 105 of the respective trajectory.
[00103] The surgery planning system 20 is configured to control the III 22 such that different types of trajectories 102, 104, 106 are output in a visually distinguishable manner, depending on whether they maintain the safety margin distance from all of the prohibited regions 93, 94. The surgery planning system 20 is configured to control the III 22 such that different trajectories 102, 104, 106 are output in a visually distinguishable manner, depending on whether they have a distance at least equal to the safety margin distance from all of the prohibited regions 93, 94. The different visually distinguishable representations may include color coded lines.
[00104] For illustration, a trajectory 102 that passes to the target 95 through the hard tissue and has a distance at least equal to the safety margin distance from all the prohibited region(s) 93, 94 may be displayed in a first style, e.g., a first color (such as green color) to indicate acceptability of the trajectory 102. A trajectory 104 that passes to the target 95 through the hard tissue and has a distance less than the safety margin distance from at least one prohibited region 93, 94 while not intersecting any of the prohibited regions 93, 94 may be displayed in a second style, e.g., a second color (such as orange color) to indicate that the safety margin distance is not complied with for at least one of the prohibited regions. A trajectory 106 that intersects at least one of the prohibited regions 93, 94 may be displayed in a third style, e.g., a third color (such as red color) to indicate non-acceptability.
[00105] Other techniques of making the trajectories visually distinguishable may be used. For illustration, different line thicknesses and/or broken line types (solid, dashed, dashed- dotted) may be used to distinguish trajectories depending on their geometrical arrangement (in particular their distances) from the one or several prohibited regions.
[00106] While the various trajectories 102, 104, 106 may be displayed concurrently on the III 22, the trajectories 102, 104, 106 may be output in a time-sequential manner. This may be done responsive to a user input, as shown in Fig. 4, Fig. 5, and Fig. 6.
[00107] Fig. 4, Fig. 5, and Fig. 6 show graphics output via the III 22 at different times. The III 22 displays an interface element 110 that is displaceable by a user relative to the graphical representation 100. While a 2D cross-sectional view is shown in Fig. 4, Fig. 5, and Fig. 6, the interface element 110 may be displaceable in 3D (e.g., along a curved surface of a skull) to specify various entry points of a trajectory.
[00108] The surgery planning system 20 is configured to determine, for every position of the interface element 110, whether or not there is an acceptable trajectory (e.g., a straight line) from the entry point of a trajectory specified by the position of the interface element 110 to the target 95. A first additional trajectory 106 is shown in a visual representation (e.g., red color) that indicates that the first additional trajectory 106 passes through at least one prohibited region 94 (Fig. 4). As the position of the interface element 110 is varied, the III 22 outputs a second additional trajectory 104 in another visual representation (e.g., orange color) that indicates that the second additional trajectory 104 does not intersect any one of the prohibited regions 93, 94 but has a distance less than the safety margin distance 96 from at least one prohibited region 94 (Fig. 5). As the position of the interface element 110 is varied again, the III 22 outputs an acceptable trajectory 102 in yet another visual representation (e.g., green color) that indicates that the acceptable trajectory 102 has a distance at least equal to the safety margin distance 96 from each of the prohibited regions 93, 94 (Fig. 6).
[00109] The graphical representation may represent a 3D-rendering of the anatomical structure. The interface element 22 may be allowed to be moved so as to mimic 3D movement, e.g., along the curved surface of a skull. This is illustrated in Fig. 7, Fig. 8, and Fig. 9.
[00110] Fig. 7, Fig. 8, and Fig. 9 show graphics output via the III 22 at different times. The III 22 displays an interface element 110 that is displaceable by a user relative to the graphical representation 100. The interface element 110 may be displaceable in 3D (e.g., along a curved surface of a skull) to specify various entry points of a trajectory, as schematically indicated in Fig. 7.
[00111 ] The surgery planning system 20 is configured to determine, for every position of the interface element 110, whether or not there is an acceptable trajectory (e.g., a straight line) from the entry point of a trajectory specified by the position of the interface element 110 to the target 95. The III 22 outputs an acceptable trajectory 102 in a visual representation (e.g., green color) that indicates that the acceptable trajectory 102 has a distance at least equal to the safety margin distance 96 from each of the prohibited regions 93, 94 (Fig. 7). As the position of the interface element 1 10 is varied, the III 22 outputs a second additional trajectory 104 in another visual representation (e.g., orange color) that indicates that the second additional trajectory 104 does not intersect any one of the prohibited regions 93, 94 but has a distance less than the safety margin distance 96 from at least one prohibited region 94 (Fig. 8). As the position of the interface element 110 is varied again, the III 22 outputs a first additional trajectory 106 in yet another visual representation (e.g., red color) that indicates that the first additional trajectory 106 passes through at least one prohibited region 94 (Fig. 9). [00112] Fig. 10 is a flow chart of a method 120. The method 120 may be performed automatically by the surgical system 1 , e.g., by the surgery planning system 20 of the surgical system 1 .
[00113] At step 121 , the surgery planning system 20 controls the III 22 to enable a user input that specifies the target 95. The III 22 is controlled such that the target 95 can be defined relative to a graphical representation of an anatomical structure (such as the skull base). The results of the processing of image data may be used in enabling the inputting of the target. For illustration, selection of a target 95 at an inwardly facing surface of the skull base may be enabled if a lateral skull base access is to be formed for removal or treatment of cancerous or diseased tissue such as tumors. The surface of the skull may be determined using image segmentation.
[00114] At step 122, the surgery planning system 20 controls the III 22 to enable a user input that specifies one or several prohibited regions 93, 94. The III 22 is controlled such that the prohibited region(s) 93, 94 can be defined relative to a graphical representation of an anatomical structure (such as the skull base). The results of the processing of image data may be used in enabling the inputting of the prohibited region(s) 93, 94, as explained in detail above.
[00115] At step 123, the surgery planning system 20 controls the III 22 to enable a user input that specifies a safety margin distance.
[00116] At step 124, the surgery planning system 20 controls the III 22 to output laser beam trajectories in a visually distinguishable manner, depending on how the laser beam trajectories are positioned relative to the prohibited regions 93, 94.
[00117] The techniques disclosed herein can be used to determine trajectories that are straight lines around which a cylindrical or frustoconical volume of hard tissue ablation extends. The systems and methods disclosed herein are also applicable to more complex geometries of trajectories.
[00118] For illustration, the systems and methods according to embodiments may be configured such that they can determine trajectories for laser radiation which include several straight segments arranged at an angle relative to each other (as explained with reference to Figure 11 ) and/or offset from each other in a direction transverse to a beam propagation direction (as explained with reference to Figure 12). [00119] Figure 11 illustrates operation of a surgery planning system or method that determines a trajectory 130 from an entry point 110 into the hard tissue to the target 95. The trajectory 130 includes two or more straight line segments 131 , 132. The two or more straight line segments 131 , 132 are arranged at an angle relative to each other.
[00120] A position of an intermediate point 133 to which two of the two or more straight line segments 131 , 132 connect may be varied. This may be done automatically by the surgery planning system to automatically determine an optimum position of the intermediate point 133 (e.g., with respect to maximizing distances from the prohibited regions). Alternatively or additionally, the surgery planning system may enable the user to manipulate the position of the intermediate point 133 via the III 22.
[00121 ] The surgery planning system may take into account the geometrical constraints when assessing whether the trajectory 130 is possible. The diameter of both a first volume of tissue ablation around a first line segment 131 (e.g., the diameter(s) of a cylindrical or frustoconical tissue ablation volume) may be taken into account to ensure that a second volume of tissue ablation around a second line segment 132 can be ablated through the first volume of tissue ablation. In other words, the surgery planning system may be configured to not only check that desired minimum distances from the one or several prohibited regions are maintained, but also to ensure that laser ablation remains possible along the various line segments that, in combination, form the trajectory 130.
[00122] Figure 12 illustrates operation of a surgery planning system or method that determines several volumes 141 -144 of tissue ablation into the hard tissue to a target. Each of the tissue ablation volumes 141 -144 may have a shape selected to be frustoconical or cylindrica. The tissue ablation volumes 141 -144 may be arranged in an abutting or overlapping manner and may be offset from each other in a direction transverse to the center axes of the tissue ablation volumes, i.e. , transverse to the laser beam propagation direction.
[00123] By combining several tissue ablation volumes 141 -144 that are arranged in an abutting or overlapping manner, a volume 140 having a non-frustoconical and non- cylindrical shape can be ablated. For illustration, the volume 140 can define a non-circular opening in hard tissue surfaces. This allows the tissue ablation volume to be tailored to the needs of the access that is formed in the respective case. [00124] This description and the accompanying drawings that illustrate aspects and embodiments of the present invention should not be taken as limiting-the claims defining the protected invention. In other words, while the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. Various mechanical, compositional, structural, electrical, and operational changes may be made without departing from the spirit and scope of this description and the claims. In some instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure the invention. Thus, it will be understood that changes and modifications may be made by those of ordinary skill within the scope and spirit of the following claims. In particular, the present invention covers further embodiments with any combination of features from different embodiments described above and below. For example, it is possible to operate the invention in an embodiment wherein:
-While color coding may be used to output different laser trajectories in a visually distinguishable manner, other visually distinguishable representations may be used.
-While the results of the surgery planning may be used to control the laser beam generator and robot arm to perform hard tissue removal in a lateral skull base, e.g. for removal or treatment of cancerous or diseased tissue such as tumors, the surgical system may be configured to perform additional or alternative procedures. For illustration, the results of the surgery planning may be used to control the laser beam generator and robot arm to perform hard tissue removal in a mastoid portion for an antrostomy, a mastoidectomy, mastoidectomy templating, a tympanotomy.
-While the trajectories may be laser beam trajectories for laser ablation of hard tissue, the surgery planning system and method may be used in association with alternative or additional hard tissue removal tools.
-While the surgical system may be implemented as a surgical robot that, once started to perform a procedure, acts with a high degree of autonomous decision making, the surgical system may also be an assistance system that assists a surgeon through part of a surgery.
[00125] The disclosure also covers all further features shown in the Figs, individually although they may not have been described in the afore or following description. Also, single alternatives of the embodiments described in the figures and the description and single alternatives of features thereof can be disclaimed from the subject matter of the invention or from disclosed subject matter. The disclosure comprises subject matter consisting of the features defined in the claims or the exemplary embodiments as well as subject matter comprising said features.
[00126] Furthermore, in the claims the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single unit or step may fulfil the functions of several features recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. The terms “essentially”, “about”, “approximately” and the like in connection with an attribute or a value particularly also define exactly the attribute or exactly the value, respectively. The term “about” in the context of a given numerate value or range refers to a value or range that is, e.g., within 20%, within 10%, within 5%, or within 2% of the given value or range. Components described as coupled or connected may be electrically or mechanically directly coupled, or they may be indirectly coupled via one or more intermediate components. Any reference signs in the claims should not be construed as limiting the scope.
[00127] A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. In particular, e.g., a computer program can be a computer program product stored on a computer readable medium which computer program product can have computer executable program code adapted to be executed to implement a specific method such as the method according to the invention. Furthermore, a computer program can also be a data structure product or a signal for embodying a specific method such as the method according to the invention.