US20220331008A1 - System and method for location determination using movement of an optical label fixed to a bone using a spatial mapping camera - Google Patents

Abstract

A system for determining a location for a surgical procedure, having a 3D spatial mapping camera, the 3D spatial mapping camera configured to map a bone. The system also includes a marker attached to a distal end section of the bone, such that the 3D spatial mapping camera is configured to capture a plurality of images of the marker as the bone is rotated in a non-linear path. The images also include data identifying a location of the marker. The system also includes a computer system that receives the data from the images captured by the 3D spatial mapping camera and determines a location of a mechanical axis of the bone, and a mixed reality display, where the computer system is configured to send the location of the mechanical axis to the mixed reality display and the mixed reality display is configured to provide a virtual display of the mechanical axis of the bone.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
• This application is a continuation of U.S. patent application Ser. No. 17/562,917, filed on Dec. 27, 2021, which is a continuation of U.S. patent application Ser. No. 17/402,360, filed on Aug. 13, 2021, which is a continuation of U.S. patent application Ser. No. 17/221,760, filed on Apr. 2, 2021, which is hereby incorporated by reference herein in its entirety, including but not limited to those portions that specifically appear hereinafter, the incorporation by reference being made with the following exception: in the event that any portion of the above-referenced application is inconsistent with this application, this application superseded said above-referenced application.
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
• Not applicable.
BACKGROUND1. The Field of the Present Disclosure
• The present disclosure relates generally to surgical systems and methods of facilitating the efficiency and accuracy of implanting surgical prostheses using fixed markers attached to an end portion of a bone, mixed reality and 3D spatial mapping devices.
2. Description of Related Art
• In conventional knee replacement procedures, a femur is cut on a distal end which can then receive a knee replacement. Conventionally, an anatomical axis of the femur is identified by inserting a rod into the femoral canal of the femur. The rod is inserted into the entire length of the femur and acts as an identifier for the anatomical axis of the femur. A mechanical axis of the femur can be estimated as shifted at about 5 degrees, or 5 degrees offset, from the anatomical axis of the rod. A cut line, or cut plane, is then identified as perpendicular to the mechanical axis. The mechanical axis is conventionally difficult to identify in the operating room because of the engagement of the proximal end of the femur in the hip socket. However, this conventional procedure, inserting a rod into the entire length of the femur, is very invasive, which can cause a variety of complications and potential injury to the patient.
• Once the mechanical axis is identified, the proper positioning of a jig or knee implant can be made. A femoral implant and tibial implant are typically designed to be surgically implanted into the distal end of the femur and the proximal end of the tibia, respectively. The femoral implant is further designed to cooperate with the tibial implant in simulating the articulating motion of an anatomical knee joint.
• These femoral and tibial implants, in combination with ligaments and muscles, attempt to duplicate natural knee motion as well as absorb and control forces generated during the range of flexion. In some instances however, it may be necessary to replace or modify an existing femoral and/or tibial implant. Such replacements are generally referred to as revision implants.
• To prepare a femur and tibia for such a knee replacement and form an engagement with femoral and tibial implants, the femur and tibia bones must be cut in very specific and precise ways and at very specific and precise angles and locations, so that the prepared bone will properly engage with and be secured to the corresponding implants. In order to make these cuts properly, a surgeon traditionally uses a jig, or surgical cutting guide as known to those skilled in the field, which can be removably attached or secured to the bone, such that slots, or guides, in the jig facilitate the precise cuts necessary to secure the corresponding implants.
• The phrase “jig” as used herein, shall thus refer broadly to a surgical cutting guide, that may be configured and arranged to be fixed or attached to a bone, or may be secured adjacent to a bone or other tissue to be cut by a surgeon and identify a relative location, angle and/or cutting plane that a surgeon should cut on the adjacent bone or tissue, as known in the art. A jig may include predetermined slots and/or cutting surfaces to identify where a surgeon should cut the adjacent bone or tissue, wherein such cuts may correspond to a shape of a surgical implant that may be attached to the cut bone or tissue.
• Therefore, there is a need for a system that can identify a mechanical axis of the femur, or the center of the femoral head, without having to insert a rod into the femur or expose the hip socket, which can lead to complication during surgery and require significant surgery time and additional recovery time for the patient.
• Accordingly, there is a need for a system and method of utilizing a virtual or holographic mechanical axis or surgical instrument that could facilitate increased accuracy and precision of required or desired bone cuts.
• The phrases “virtual jig,” “holographic jig,” “virtual axis,” or “holographic axis” as used herein, shall thus refer broadly to any visual rendering or projection representing an actual physical jig, or mechanical or anatomical axis of a bone, as the case may be, having some, all, or mostly all, of the same visual characteristics of the physical jig or axis, as the case may be, including the visual appearance of the same size and shape as the physical objects being virtually represented, as known in the art.
• The features and advantages of the present disclosure will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by the practice of the present disclosure without undue experimentation. The features and advantages of the present disclosure may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base, or were common general knowledge in the field relevant to the present disclosure as it existed before the priority date of each claim of this application.
BRIEF DESCRIPTION OF THE DRAWINGS
• The features and advantages of the disclosure will become apparent from a consideration of the subsequent detailed description presented in connection with the accompanying drawings in which:
• FIG. 1ais a schematic rendering of a mixed reality system of the present disclosure;
• FIG. 1bis a schematic rendering of a view through a mixed reality display of an embodiment of the present disclosure;
• FIG. 1cis a schematic rendering of a view through a mixed reality display of another embodiment of the present disclosure;
• FIG. 1dis a schematic rendering of a view through a mixed reality display of a further embodiment of the present disclosure;
• FIG. 2 is a perspective view of a 3D spatial mapping camera of another embodiment of the present disclosure;
• FIG. 3 is a perspective view of a 3D spatial mapping camera of another embodiment of the present disclosure;
• FIG. 4 is a perspective view of a 3D spatial mapping camera of another embodiment of the present disclosure;
• FIG. 5 is a perspective view of a 3D spatial mapping camera of another embodiment of the present disclosure;
• FIG. 6 is a perspective view of a 3D spatial mapping camera of another embodiment of the present disclosure;
• FIG. 7 is a perspective view of a 3D spatial mapping camera of another embodiment of the present disclosure;
• FIG. 8 is a perspective view of a 3D spatial mapping camera of another embodiment of the present disclosure;
• FIG. 9 is a perspective view of a 3D spatial mapping camera of another embodiment of the present disclosure;
• FIG. 10 is a perspective view of another embodiment system of the present disclosure;
• FIG. 11 is a perspective view of an optical label of another embodiment of the present disclosure; and
• FIG. 12 is a perspective view of another embodiment of the present disclosure.
DETAILED DESCRIPTION
• For the purposes of promoting an understanding of the principles in accordance with the disclosure, reference will now be made to the embodiments illustrated in the drawings and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the disclosure is thereby intended. Any alterations and further modifications of the inventive features illustrated herein, and any additional applications of the principles of the disclosure as illustrated herein, which would normally occur to one skilled in the relevant art and having possession of this disclosure, are to be considered within the scope of the disclosure claimed.
• It must be noted that, as used in this specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise.
• In describing and claiming the present disclosure, the following terminology will be used in accordance with the definitions set out below.
• As used herein, the terms “comprising,” “including,” “containing,” “characterized by,” and grammatical equivalents thereof are inclusive or open-ended terms that do not exclude additional, unrecited elements or method steps.
• As used herein, the terms “virtual” and “hologram” are used interchangeably, and grammatical equivalents thereof are inclusive or open-ended terms that do not exclude additional, unrecited elements or method steps. These terms are used to describe visual representations of an actual physical object, device or element, having some, all, or mostly all, of the same visual characteristics of the physical device, including the visual appearance of the same size and shape as the physical object device or element being virtually represented.
• Applicant has discovered a novel system and method for generating and using a virtual axis and/or virtual instrument, in a surgical procedure, for example, in a knee or tibial implant procedure, or other desired surgical procedure.
• The phrase “virtual system” as used herein, shall refer broadly to any system capable of generating or creating a simulated or virtual rendering or projection of physical or structural objects, devices or features virtually identical or, substantially identical to an actual physical device, object, instrument or other structure, as known in the art. A virtual system may also include a device, mechanism, or instrument capable of projecting or displaying the desired simulated or virtual rendering or projection of physical or structural features virtually identical or substantially identical to an actual physical device or features or portions thereof. A virtual system may also enable a user to manipulate, move and/or modify the simulated or virtual rendering or projection.
• The phrase “mixed or augmented reality system” as used herein, shall refer broadly to any system capable of generating or creating a simulated or virtual rendering or projection of physical or structural objects, devices or features virtually identical or substantially identical to an actual physical device, instrument or other structure or feature or portions thereof, as known in the art. A mixed or augmented reality system may also include a device, mechanism, or instrument capable of projecting or displaying the desired a simulated or virtual rendering or projection of physical or structural features virtually identical or substantially identical to an actual physical device overlaid or concurrently with actual physical structures, objects, mechanisms or devices in reality, thus incorporating the virtual rendering or projection in real world settings with actual physical elements. A mixed or augmented reality system may also enable a user to manipulate, move and/or modify the simulated or virtual rendering or projection.
• The phrase “mixed or augmented reality instrument” as used herein, shall refer broadly to any device, mechanism or instrument used in a mixed or augmented reality system, including a device capable of generating or creating a simulated or virtual rendering or projection of physical or structural elements and/or features virtually identical or substantially identical to an actual physical device, object, instrument or other physical structure, as known in the art. A mixed or augmented reality instrument may also be capable of projecting or displaying the desired simulated or virtual rendering or projection of physical or structural features or elements virtually identical or substantially identical to an actual physical device or object overlaid or concurrently with actual physical structures, object, mechanism or devices in reality, thus incorporating the virtual rendering or projection in real world settings with actual physical elements or features. A mixed or augmented reality instrument may also enable a user to manipulate, move and/or modify the simulated or virtual rendering or projection.
• The phrase “holographic representation” as used herein, shall refer broadly to a visual rendering or projection representing an actual physical device, object or element or portion thereof, having some, all, or mostly all, of the same visual characteristics of the corresponding physical device, object or element, including the visual appearance of the same size and shape as the physical objects being virtually represented, as known in the art.
• Referring to the drawings, where like numbers represent like elements,FIG. 1a, in a disclosed embodiment, illustrates a mixed or augmented reality system, generally indicated at100, which can be used to produce, or display, a desired mixed or augmented reality instrument, such as an anatomical or mechanical axis of a bone in a display to a surgeon or user, or stated another way, that is visible and manipulatable by a surgeon or user. The mixed oraugmented reality system100 may also enable a user to activate or deactivate, in full or in part, the marker, such as an axis, or virtual instrument(s), making a virtual axis appear or disappear, as desired in a mixed reality assisted surgery, for example.
• The mixed oraugmented reality system100 may include a mixed oraugmented reality headset102 which may include a transparent or mostlytransparent viewer104, or display, which can be suspended or positioned in front of a user's eyes. In alternative embodiments, theviewer104, or mixed reality display, may be a stand-alone device detached and separate from the surgeon, or may be mounted in another desired location in an operating room or other desired location. Theheadset102 may include aheadband106 attached to theviewer104, which may be used to secure theheadset102 to a user'shead108, thereby securing theviewer104 in place in front of the user's eyes.
• Thetransparent viewer104 may be configured to project, or otherwise make viewable, on an interior surface of theviewer104, a holographic virtual image or images, such as a virtual instrument, for example, an anatomical or mechanical axis of a bone, which may be positionally manipulated by the user, surgeon, third party or remote system, such as a remote computer system. For the purpose of this disclosure the term “mechanical axis” as used herein shall be defined broadly in reference to a bone having a proximal joint and a distal joint, as a straight line connecting the joint center points of the proximal and distal joints in the frontal or sagittal planes. Theheadset102 may be configured to view holographic images or, alternatively, the holographic images may be turned off and the user wearing theheadset102 may be able to view the surrounding environment through thetransparent viewer104, unobstructed. As such, a user, such as a surgeon for example, can wear the mixed oraugmented reality headset102 and then can choose to activate a holographic image to aide in facilitating a surgical procedure and then shut off the holographic image in order to perform the surgical procedure un-obscured, visually.
• One embodiment of the disclosedheadset102 may be a product created and manufactured by Microsoft, known as the HoloLens® mixed or augmented reality system, or any suitable mixed or augmented reality system for generating virtual images viewable by a user or surgeon.Headset102 may be a conventional “off the shelf” product with a built-in platform that enables all of the features described herein with respect to theheadset102.
• In addition to identifying axes of a bone, in alternative embodiments, theheadset102, such as a Microsoft HoloLens product, can be loaded or preloaded with all desired or required virtual instruments, including virtual jigs or surgical cutting guides, virtual drill bits, and/or a virtual target which can identify relative locations of a plurality of holes to be drilled by a surgeon to facilitate the fastening of a jig or other device onto a desired bone at the proper desired location, and any other desired virtual instruments or holograms. The Microsoft HoloLens product and its capabilities and features, or any suitable mixed or augmented reality system such as is described herein with respect to theheadset102, are known to those skilled in the art.
• Themixed reality system100 may also include a computer orcomputer system200 having enabling software to communicate with theheadset102, by both receiving information from theheadset102 and transmitting data and virtual images to theheadset102. It is therefore to be understood, by way of the circuit diagram and dashed lines shown inFIG. 1, thatheadset102 is electronically connected to thecomputer system200 and a 3D spatial mapping device orcamera300. The 3Dspatial mapping camera300 is electronically connected to theheadset102 and thecomputer system200, as represented by the dashed lines shown inFIG. 1. While the 3Dspatial mapping camera300 is electronically connected to theheadset102, the 3Dspatial mapping camera300 may be separate from and not mechanically connected to theheadset102.
• Themixed reality system100 may also include a 3Dspatial mapping camera300. One embodiment of the disclosedspatial mapping camera300 may be a product created and manufactured by Microsoft, known as the Azure Kinect®, or any suitable 3D spatial mapping camera or LiDAR Scanner capable of continuous 3D mapping and transition corresponding 3D images, such as bones, anatomy, or other desired 3D objects. Thespatial mapping camera300 may be a conventional “off the shelf” product with a built-in platform that enables all of the features described herein with respect to thespatial mapping camera200. Furthermore, thespatial mapping camera200, such as a Microsoft Azure Kinect product, can be loaded or preloaded with all necessary software to enable wireless communication between thespatial mapping camera300 and thecomputer system200 and/or theheadset102. The Microsoft Azure Kinect product and its capabilities and features, or any suitable 3D spatial mapping camera such as is described herein with respect to thespatial mapping camera300, are known to those skilled in the art.
• Theheadset102,computer system200 andspatial mapping camera300, may be programmed and configured to enable asurgeon107 to see and manipulate a virtual, or holographic target or axis, with respect a patient'sbone400, anatomical, or any other desired location, which may receive a surgical implant. Theheadset102,computer system200 andspatial mapping camera300 may communicate with one another via a local network connection, W-Fi, Bluetooth® wireless technology, or any other known wireless communication signal.
• Specifically, thespatial mapping camera300, that may be programed to communicate with thecomputer system200 having enabling software, may utilize such enabling software to map thebone400 and generate map data representative of a three dimensional map of a surface of thebone400, or other desired anatomy, to help identify the location and orientation of an anatomical or mechanical axis of a bone, which can then facilitate the proper placement of a jig, implant or other device, to thebone400, prior to cutting the knee.
• Themixed reality system100 may also include amarker500. Themarker500 may be attached or otherwise fixed to a distal end of thebone400, which may be exposed. While a plurality ofmarkers500 may be used, if desired, a single,exclusive marker500, may also be used in this disclosed embodiment.Bone400 is a femur in the illustrated embodiment ofFIG. 1, however, themarker500 may be fixed to any desired bone having in inaccessible pivot point or fulcrum. The “distal end section or portion” may include the terminal distal end, the distal end section or portion as used herein being defined broadly as a distal half of thebone400. Themarker500 may include a scannable, visual,optical label502, such as a QR code.
• The surgeon may attach themarker500 to an exposed distal end section of thebone400, at a predetermined or desired location. Thespatial mapping camera300 may spatially map themarker500 and the exposedbone400 to map the surface of the exposed bone and relative location of themarker500, including theoptical label502.
• Theoptical label502 may include a QR code or other optical identifiers, such as shapes having distinct corners, which can be identified by the 3Dspatial mapping camera300 and the location of the optical label identified and stored.
• The surgeon may then rotate thebone400, with themarker500 fixed there to, in anon-liner path504, such as a circular path, ovular path or another desired, predetermined, or random non-liner path. Thebone400 may be rotated about a fulcrum or pivot point that is not exposed to the surgeon's view, for example the socket joint of the hip, but may also include any desired pivot point or fulcrum that may not be exposed to the surgeon, such as a shoulder joint, for example.
• At specific time intervals, the 3Dspatial mapping camera300 may capture an image of the location of themarker500 andoptical label502 and store the location data. The time intervals between the plurality of captures of the location data may be predetermined and consistent time intervals, at random time intervals, or at time intervals controlled by the surgeon. Additionally, the number of image captures by the 3D spatial mapping camera may be set at a predetermined number, or at a random number, or at a number that is determined by the surgeon. For example, the 3D spatial mapping camera may capture an image every second for 30 seconds, or may capture 10 images every second for 3 seconds, or any other desired number of images over any desired time intervals.
• After the plurality of image captures of themarker500 andoptical label502, the images and location data may be sent to the computer system20 which may then process the image and location data, and using a preloaded algorithm, may generate themechanical axis402 of thebone400.
• Additionally, thecomputer system200, may also process the image and location data, and using a preloaded algorithm, may generate amechanical center404 of the femoral head of thebone400.
• Thecomputer system200, may also process the image and location data, and using a preloaded algorithm, may generate acut line406, or cut plane, that can be used by a surgeon to identify where thebone400 should be cut in preparation of the attachment of an implant or other desired surgical device. Thecut plane406 forms an angle with themechanical axis402 of thebone400 and may, for example, be oriented orthogonal to the mechanical axis of the bone.
• Thecomputer system200 may then send data related to themechanical axis402,mechanical center404, cutting line orplane406, and/orvirtual jig410, to theheadset102, which can then utilize the data provided by the computer to provide a virtual image of themechanical axis402,mechanical center404, the cutting line orplane406, and/orvirtual jig410, relative to thebone400, which can then be displayed on theviewer104, that visually appears tooverlay bone400 at a position and in an orientation of themechanical axis402 when viewed with the mixed reality display. The surgeon can then view the virtualmechanical axis402,mechanical center404, the cutting line orplane406, and/orvirtual jig410, in their accurate location with respect to thephysical bone400, which can be seen simultaneously with the virtual images. The surgeon can then cut thebone400, attach a device or implant to thebone400 or otherwise manipulate the bone using the accurately displayed virtualmechanical axis402,mechanical center404, and/or the cutting line orplane406.
• FIG. 1billustrates a schematic view of a perspective through theviewer104 of themixed reality headset102, representing the view of a wearer of themixed reality headset100 during a procedure. As shown, the view can display a virtualmechanical axis402 overlaid on the actualphysical bone400, enabling the wearer to see or view the virtualmechanical axis402 simultaneously with thebone400,marker500 andlabel502.
• FIG. 1cillustrates a schematic view of a perspective through theviewer104 of themixed reality headset102, representing the view of a wearer of themixed reality headset100 during a procedure. As shown, the view can display a virtualmechanical axis402 overlaid on the actualphysical bone400, enabling the wearer to see or view the virtual cutting line orplane406 simultaneously with thebone400,marker500 andlabel502.
• FIG. 1dillustrates a schematic view of a perspective through theviewer104 of themixed reality headset102, representing the view of a wearer of themixed reality headset100 during a procedure. As shown, the view can display a virtualmechanical axis402 overlaid on the actualphysical bone400, enabling the wearer to see or view avirtual jig410 simultaneously with thebone400,marker500 andlabel502.
• In another embodiment, as shown inFIG. 2, the 3D spatial mapping camera may be fixed or removably attached to astand600. Thestand600 may be anchored or fixed to a desired location in the operating room, or the stand may be mobile, utilizing a plurality ofwheels602, for example.
• In another embodiment, as shown inFIG. 3, the 3Dspatial mapping camera300 may be fixed or removably attached to astand700. Thestand700 may be anchored or fixed to a desired location in the operating room, or the stand may be mobile, utilizing a plurality ofwheels702, for example. Thestand700 may also include aflexible arm704 that may allow a user to manipulate the positioning of the 3Dspatial mapping camera300.
• In another embodiment, as shown inFIG. 4, the 3Dspatial mapping camera300 may be sized and configured to be fixed or removably attached to astrap800. Thestrap800 may be adjustable and can be configured to be worn by a user around their head, chest or other desired part of the body.
• In another embodiment, as shown inFIG. 5, the 3Dspatial mapping camera300 may be sized and configured to be fixed or removably attached to aheadset900. Theheadset900 may be worn on the head of a user as part of eye protection or glasses or as part of a mixed reality headset.
• In another embodiment, as shown inFIG. 6, the 3Dspatial mapping camera300 may be sized and configured to be attached or integrated into alight source panel1000 used during an operation. The illustration ofFIG. 6 shows different locations on thelight source panel1000 that could incorporate the 3D spatial mapping camera300 (various location identified by the arrows). For example, the 3D spatial mapping camera may be incorporated into ahandle1002, alight source1004, aframe1006, or asleeve108 which may provide a sterile barrier for thehandle1002.
• In another embodiment, as shown inFIG. 7, the 3Dspatial mapping camera300 may be fixed or removably attached to an operating table1100 via amount1102. Themount1102 may be fixed or attached to the operating table at any desired location and may be adjustable in height and/or direction, enabling a surgeon or user to manipulate the angle of view and location of the 3Dspatial mapping camera300.
• In another embodiment, as shown inFIG. 8, the 3Dspatial mapping camera300 may be worn by asurgeon1200 or anassistant1202 using any of the aforementioned 3D spatial mapping camera embodiments. The 3D spatial mapping camera may be worn by a user inside or outside of a sterile field and could be worn in a way that would conform to specific procedural requirements a particular surgery.
• In another embodiment, as shown inFIG. 9, the 3Dspatial mapping camera300 may be fixed or removably attached to a variety of different locations within anoperating room1300. For example, the 3Dspatial mapping camera300 may be attached or fixed to awall1302, afloor1304, awall panel1306, awall fixture1308, aceiling1310 or any other desired location. Additionally, multiple 3Dspatial mapping cameras300 can be used simultaneously or in concert with one another where each 3Dspatial mapping camera300 may be mounted in different locations in theoperating room1300 or at the same location.
• In another embodiment, as shown inFIG. 10, thesystem100 may include all of the same components as discuss and disclosed inFIG. 1, but may also include a plurality ofmarkers500 attached to the exposed portion of thebone400. As disclosed above with respect to asingle marker500, the use ofmultiple markers500 may provide additional data that may be captured by the 3Dspatial mapping camera300 and processed by thecomputer system200 to locate themechanical axis402,mechanical center404, and/or the cut line orplane406.
• In another embodiment, as shown inFIG. 11, the earlier disclosedoptical labels502 may be QR codes, as shown in examples1400,1402, and1404.QR codes1400,1402, and1404 may include various shapes and be of various desired sizes, and may include a plurality of corners, or sharp points that may be scanned their images captured by the 3Dspatial mapping camera300 to provide additional location data that may be processed by thecomputer system200 as discussed in the other disclosed embodiments. Additionally, themarkers500 and/oroptical labels502 may be may of reflective material, of any desired color, may emit light, may be formed of any desired shape or of any desired size, to facilitate the capturing of an image to provide location data.
• FIG. 12 illustrates that, additional uses of the disclosedsystem100 and accompanying embodiments, may include abone1500 having afracture1501. By using the above-mentioned system and method a surgeon may identify the location of thefracture1501, being a fulcrum or pivot point as described above, or other discontinuity. Thesystem100 can then be used, using the same location determining methodology and system described above, to locate the location of a stabilizingrod1502 and the corresponding location of wherescrews1504 would need to be located to be properly and accurately received by therod1502. This can be particularly useful in circumstances where critical locations are not visibly exposed to the surgeon.
• It is to be understood that the various embodiments disclosed and described above and shown in the accompanying figures, may be interchangeably used together, independently or in any desired combination of disclosed features.
• In the foregoing Detailed Description, various features of the present disclosure are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed disclosure requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the following claims are hereby incorporated into this Detailed Description of the Disclosure by this reference, with each claim standing on its own as a separate embodiment of the present disclosure.
• It is to be understood that the above-described arrangements are only illustrative of the application of the principles of the present disclosure. Numerous modifications and alternative arrangements, including but not limited to combinations of elements and/or features from the various disclosed embodiments may be devised by those skilled in the art without departing from the spirit and scope of the present disclosure and the invention is intended to cover such modifications, arrangements and combinations. Thus, while the present disclosure has been shown in the drawings and described above with particularity and detail, it will be apparent to those of ordinary skill in the art that numerous modifications, including, but not limited to, variations in size, materials, shape, form, function and manner of operation, assembly and use may be made without departing from the principles and concepts set forth herein.

Claims (42)

1. A system for determining a location for a surgical procedure, utilizing:

a 3D spatial mapping device, wherein the 3D spatial mapping device is configured to generate map data representative of a three dimensional map a surface of a bone,

a marker attached to a distal end portion of the bone;

wherein the 3D spatial mapping device is configured to capture a plurality of images of the marker as the bone is rotated in a non-linear path, wherein the images include marker data identifying a marker location of the marker relative to the three dimensional map data of the surface of the bone;

a computer system that receives the map data and marker data from the images captured by the 3D spatial mapping device and determines a position and orientation of a mechanical axis of the bone relative to the bone;

a mixed reality display, wherein the computer system is configured to send the position and orientation of the mechanical axis to the mixed reality display and the mixed reality display is configured to provide a virtual image of the mechanical axis of the bone that visually appears to overlay the bone at the position and in the orientation of the mechanical axis when viewed with the mixed reality display.

2. The system ofclaim 1, wherein the mixed reality display is a mixed reality headset.

3. The system ofclaim 1, wherein the distal end section of the bone is visually exposed.

4. The system ofclaim 1, wherein the marker includes a QR code.

5. The system ofclaim 1, wherein the system includes only a single marker.

6. The system ofclaim 1, wherein the computer system that receives the data from the images captured by the 3D spatial mapping device and determines a location of a mechanical center of a head of the bone.

7. The system ofclaim 1, wherein the computer system that receives the data from the images captured by the 3D spatial mapping device and determines a location of a cut line or cut plane.

8. The system ofclaim 1, wherein the 3D spatial mapping device is configured to capture a plurality of images of the marker as the bone is rotated in a non-linear path about a pivot point, or fulcrum, that is not visual exposed.

9. The system ofclaim 1, wherein the system may include a plurality of markers attached to the bone.

10. The system ofclaim 1, wherein the 3D spatial mapping device is configured to capture the plurality of images at predetermined time intervals during rotation of the bone.

11. A system determining a location for a surgical procedure, comprising:

a 3D spatial mapping device, wherein the 3D spatial mapping device is configured to map a surface of a bone,

a marker, comprised of a QR code, attached to a distal end portion of the bone; wherein the 3D spatial mapping device is configured to capture a plurality of images of the marker relative to the bone as the bone moved relative to the 3D spatial mapping device, wherein the images include marker data identifying a location of the marker relative to the bone;

a computer system that receives the marker data from the images captured by the 3D spatial mapping device and determines a position and orientation of a mechanical axis of the bone relative to the bone;

a mixed reality display, wherein the computer system is configured to send the position and orientation of the mechanical axis relative to the bone to the mixed reality display and the mixed reality display is configured to provide a virtual image of the mechanical axis of the bone that visually appears to overlay the bone at the position and in the orientation of the mechanical axis when viewed with the mixed reality display.

12. The system ofclaim 11, wherein the mixed reality display is a mixed reality headset.

13. The system ofclaim 11, wherein the distal end section of the bone is visually exposed.

14. The system ofclaim 11, wherein the movement of the bone occurs along a non-linear path.

15. The system ofclaim 11, wherein the system includes only a single marker.

16. The system ofclaim 11, wherein the computer system that receives the data from the images captured by the 3D spatial mapping device and determines a location of a mechanical center of a head of the bone.

17. The system ofclaim 11, wherein the computer system that receives the data from the images captured by the 3D spatial mapping device and determines a location of a cut line or cut plane.

18. The system ofclaim 11, wherein the 3D spatial mapping device is configured to capture a plurality of images of the marker as the bone is rotated in a non-linear path about a pivot point, or fulcrum, that is not visual exposed.

19. The system ofclaim 11, wherein the system may include a plurality of markers attached to the bone.

20. The system ofclaim 11, wherein the 3D spatial mapping device is configured to capture the plurality of images at predetermined time intervals during rotation of the bone.

21. A system for determining a location for a surgical procedure, utilizing:

a 3D spatial mapping camera, wherein the 3D spatial mapping camera is configured to map a bone, a marker attached to a distal end section of the bone;

wherein the 3D spatial mapping camera is configured to capture a plurality of images of the marker as the bone is moved, wherein the images include data identifying a location of the marker;

a computer system that receives the data from the images captured by the 3D spatial mapping camera and determines a location of a cut plane on the bone; and

a mixed reality display, wherein the computer system is configured to send the location of the mechanical axis to the mixed reality display and the mixed reality display is configured to provide a virtual image of the cut plane that visually appears to overlay the bone at a position and in an orientation relative to the bone when viewed with the mixed reality display.

22. The system ofclaim 21, wherein the mixed reality display is a mixed reality headset.

23. The system ofclaim 21, wherein the distal end section of the bone is visually exposed.

24. The system ofclaim 21, wherein the marker includes a QR code.

25. The system ofclaim 21, wherein the system includes only a single marker.

26. The system ofclaim 21, wherein the computer system that receives the data from the images captured by the 3D spatial mapping camera and determines a location of a mechanical center of a head of the bone.

27. The system ofclaim 21, wherein the computer system that receives the data from the images captured by the 3D spatial mapping camera and determines a location of a mechanical axis of the bone.

28. The system ofclaim 21, wherein the 3D spatial mapping camera is configured to capture a plurality of images of the marker as the bone is rotated in a non-linear path about a pivot point, or fulcrum, that is not visual exposed.

29. The system ofclaim 21, wherein the system may include a plurality of markers attached to the bone.

30. (canceled)

31. (canceled)

32. (canceled)

33. (canceled)

34. (canceled)

35. (canceled)

36. (canceled)

37. (canceled)

38. (canceled)

39. (canceled)

40. (canceled)

41. (canceled)

42. A system for determining a location for a surgical procedure, utilizing:

a 3D spatial mapping camera, wherein the 3D spatial mapping camera is configured to map a bone,

a single marker having a QR code attached to a distal end section of the bone, wherein the distal end section of the bone is visually exposed;

wherein the 3D spatial mapping camera is configured to capture a plurality of images of the marker as the bone is rotated in a non-linear path about a pivot point, or fulcrum, that is not visual exposed, wherein the images include data identifying a location of the marker, and wherein the 3D spatial mapping camera is configured to capture the plurality of images at predetermined time intervals during rotation of the bone;

a computer system that receives the data from the images captured by the 3D spatial mapping camera and determines a location of a cut plane on the bone; and

a mixed reality display, the mixed reality display being a mixed reality headset, wherein the computer system is configured to send the location of the mechanical axis to the mixed reality display and the mixed reality display is configured to provide a virtual display of the mechanical axis of the bone, and wherein the computer system also determines a location of a mechanical center of a head of the bone.

Images