US20050020878A1 - Endoscope - Google Patents
EndoscopeDownload PDFInfo
- Publication number
- US20050020878A1 US20050020878A1US10/498,155US49815504AUS2005020878A1US 20050020878 A1US20050020878 A1US 20050020878A1US 49815504 AUS49815504 AUS 49815504AUS 2005020878 A1US2005020878 A1US 2005020878A1
- Authority
- US
- United States
- Prior art keywords
- image
- subject
- endoscope
- constructing
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00147—Holding or positioning arrangements
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00043—Operational features of endoscopes provided with output arrangements
- A61B1/00045—Display arrangement
- A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/042—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by a proximal camera, e.g. a CCD camera
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/267—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the respiratory tract, e.g. laryngoscopes, bronchoscopes
- A61B1/2676—Bronchoscopes
Definitions
- the present inventionrelates to an endoscope device, or more particularly, an endoscope device capable of navigating an endoscope to help insert the endoscope into an intracorporeal lumen, for example, the bronchi.
- an X-ray computed tomography (CT) deviceis used to produce tomographic images of a subject so as to construct three-dimensional image data representing the interior of the subject.
- the three-dimensional image datais used to diagnose a lesion.
- One of such three-dimensional imagesis a three-dimensional image representing the bronchi in the lungs.
- the three-dimensional image of the bronchiis utilized in order to three-dimensionally grasp the position of an abnormality that is suspected to be, for example, a pulmonary carcinoma.
- a bronchoscopeis inserted in order to collect a sample of a tissue using a biopsy needle located in the distal section of the endoscope.
- FIG. 31shows a bronchi 500 .
- an intracorporeal lumen that branches out in multiple stageswhen the position of an abnormality is close to the terminal of a branch, it is hard to lead the distal end of an endoscope to a target region accurately for a short period of time.
- Japanese Unexamined Patent Application Publication No. 2000-135215has proposed a system for navigating a bronchoscope to a target region.
- a three-dimensional image of an intracorporeal lumen of a subjectis constructed based on image data acquired from a three-dimensional field of the subject.
- the course to a target point along the lumenis determined using the three-dimensional image.
- Virtual endoscopic images representing regions of the lumen that lie along the courseare constructed based on the image data, and displayed in order to navigate a bronchoscope.
- the navigation to a target region to be performed in the system described in the Japanese Unexamined Patent Application Publication No. 2000-135215includes display of a live endoscopic image produced by the bronchoscope.
- virtual endoscopic images representing branch points of bronchiare displayed for the purpose of guiding the bronchoscope into a destination of insertion.
- the bronchibranch out in multiple stages.
- images constructed with the bronchoscope located in the respective bronchiare images representing a plurality of branch destinations and resembling one another.
- An object of the present inventionis to provide an endoscope device capable of reliably navigating an endoscope to a target region using guide images that represent actual branch positions.
- An endoscope device in accordance with the present inventioncomprises: three-dimensional image constructing means for constructing a three-dimensional image of an intracorporeal lumen of a subject on the basis of images representing a three-dimensional field of the subject; an endoscope that picks up images of the intracorporeal lumen of the subject; and navigation image constructing means that constructs a navigation image which includes the endoscopic image of the intracorporeal lumen of the subject that is produced by the endoscope and the three-dimensional image. While presenting the course of the endoscope to be steered in order to insert the endoscope into the intracorporeal lumen of the subject, the endoscope device helps observe or treat the subject.
- the navigation image constructing meansconstructs the navigation image adding reduced images of three-dimensional images, which represent all branch points at which the intracorporeal lumen of the subject branches out, added thereto. Owing to these constituent features, the endoscope can be reliably navigated to reach a target region using guide images representing the actual branch positions.
- FIG. 1shows the configuration of an endoscope device in accordance with an embodiment of the present invention
- FIG. 2shows the configuration of an input unit shown in FIG. 1 ;
- FIG. 3is a flowchart describing the flow of constructing navigation data to be executed by a bronchoscope navigation device shown in FIG. 1 ;
- FIG. 4is a first diagram showing a route determination screen in which the execution of the process described in FIG. 3 is performed;
- FIG. 5is a second diagram showing the route determination screen in which the execution of the process described in FIG. 3 is performed;
- FIG. 6is a flowchart describing the flow of determining a route
- FIG. 7is a first diagram showing a route determination screen in which the execution of the process described in FIG. 6 is performed;
- FIG. 8is a second diagram showing the route determination screen in which the execution of the process described in FIG. 6 is performed;
- FIG. 9is a third diagram showing the route determination screen in which the execution of the process described in FIG. 6 is performed.
- FIG. 10is an explanatory diagram concerning a variant of route search to be executed by the bronchoscope navigation device shown in FIG. 1 ;
- FIG. 11is a first flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown in FIG. 1 ;
- FIG. 12is a first diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 13is a second diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 14is a third diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 15is a fourth diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 16is a fifth diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed;
- FIG. 17is a second flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown in FIG. 1 ;
- FIG. 18is a first diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed;
- FIG. 19is an explanatory diagram concerning a motion picture to be displayed by executing the process described in FIG. 17 ;
- FIG. 20is a second diagram showing the navigation screen in which the execution of the process described in FIG. 17 is performed;
- FIG. 21is a third diagram showing the navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 22is a third diagram showing the navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 23shows a first variant of the navigation screen shown in FIG. 14 ;
- FIG. 24is a second variant of the navigation screen shown in FIG. 14 ;
- FIG. 25is a third variant of the navigation screen shown in FIG. 14 ;
- FIG. 26is a fourth variant of the navigation screen shown in FIG. 14 ;
- FIG. 27is a fifth variant of the navigation screen shown in FIG. 14 ;
- FIG. 28is a sixth variant of the navigation screen shown in FIG. 14 ;
- FIG. 29is a seventh variant of the navigation screen shown in FIG. 14 ;
- FIG. 30is an eighth variant of the navigation screen shown in FIG. 14 ;
- FIG. 31shows the structure of bronchi.
- FIG. 1 to FIG. 30are concerned with an embodiment of the present invention.
- FIG. 1shows the configuration of an endoscope device.
- FIG. 2shows the configuration of an input unit shown in FIG. 1 .
- FIG. 3is a flowchart describing the flow of constructing navigation data to be performed by a bronchoscope navigation device shown in FIG. 1 .
- FIG. 4is a first diagram showing a route determination screen in which the execution of the process described in FIG. 3 is performed.
- FIG. 5is a second diagram showing the route determination screen in which the execution of the process described in FIG. 3 is performed.
- FIG. 6is a flowchart describing the flow of route determination to be executed at a step in the process described in FIG. 3 .
- FIG. 1shows the configuration of an endoscope device.
- FIG. 2shows the configuration of an input unit shown in FIG. 1 .
- FIG. 3is a flowchart describing the flow of constructing navigation data to be performed by a bronchoscope navigation device shown in
- FIG. 7is a first diagram showing a route determination screen in which the execution of the process described in FIG. 6 is performed.
- FIG. 8is a second diagram showing the route determination screen in which the execution of the process described in FIG. 6 is performed.
- FIG. 9is a third diagram showing the route determination screen in which the execution of the process described in FIG. 6 is performed.
- FIG. 10is an explanatory diagram concerning a variant of route search to be executed by the bronchoscope navigation device shown in FIG. 1 .
- FIG. 11is a first flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown in FIG. 1 .
- FIG. 12is a first diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 13is a second diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 14is a third diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 15is a fourth diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 16is a fifth diagram showing a navigation screen in which the execution of the process described in FIG. 11 is performed.
- FIG. 17is a second flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown in FIG. 1 .
- FIG. 18is a first diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 19is an explanatory diagram concerning a motion picture to be displayed by executing the process described in FIG. 17 .
- FIG. 20is a second diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 21is a third diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 22is a third diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 23shows a first variant of the navigation screen shown in FIG. 14 .
- FIG. 24shows a second variant of the navigation screen shown in FIG. 14 .
- FIG. 25is a third variant of the navigation screen shown in FIG. 14 .
- FIG. 26shows a fourth variant of the navigation screen shown in FIG. 14 .
- FIG. 20is a second diagram showing a navigation screen in which the execution of the process described in FIG. 17 is performed.
- FIG. 21is a third diagram showing a navigation screen in which the execution of the
- FIG. 27is a fifth variant of the navigation screen shown in FIG. 14 .
- FIG. 28is a sixth variant of the navigation screen shown in FIG. 14 .
- FIG. 29is a seventh variant of the navigation screen shown in FIG. 14 .
- FIG. 30is an eighth variant of the navigation screen shown in FIG. 14 .
- an endoscope device 1 in accordance with the present embodimentcomprises: a bronchoscope 2 that is inserted into the patient's bronchi in order to pick up images of the interiors of the bronchi or to biopsy a tissue of a lesion at the terminal of a bronchus; a degree-of-insertion detection unit 3 that detects a degree of insertion of the insertion unit of the bronchoscope 2 into the patient's body and that includes, for example, an encoder; an input unit 4 formed with a flexible sheet that realizes a plurality of switches in the form of a film and that is disposed on a proximal end of the bronchoscope 2 ; and a bronchoscope navigation device 6 that constructs virtual endoscopic images (hereinafter, VBS images) representing the interiors of bronchi on the basis of CT image data, synthesizes an endoscopic image (hereinafter, a live image), which is produced by the bronchoscope 2 , with the VBS images according to an
- the input unit 4is formed on the proximal end of the bronchoscope 2 .
- the input unit 4may be formed with a footswitch.
- the bronchoscope navigation device 6comprises: a CT image data fetching unit 11 that fetches three-dimensional image data, which is constructed in a known CT system that is not shown and that produces X-ray tomographic images of a patient, via a transportable storage medium, for example, a magneto-optical (MO) disk or a digital versatile disk (DVD); a CT image data storage unit 12 in which three-dimensional image data fetched by the CT image data fetching unit 11 is stored; an MPR image construction unit 13 that constructs an MPR image on the basis of the three-dimensional image data stored in the CT image data storage unit 12 ; a route determination unit 14 that constructs a route determination screen that will be described later and that contains an MPR image constructed by the MPR image constructing unit, and determines a navigation route (hereinafter, simply, a route) along which the bronchoscope 2 is navigated through bronchi; a VBS image construction unit 15 that constructs VBS images, which represent successive regions along the route determined by the route determining unit
- the CT image data storage unit 12 and VBS image storage unit 16may be realized with one hard disk, and the MPR image construction unit 13 , route determination unit 14 , VBS image construction unit 15 , and image processing unit 17 may be realized with one arithmetic circuit.
- the CT image data fetching unit 11fetches CT image data from a transportable storage medium such as a MO disk or a DVD.
- the CT image data fetching unit 11may be realized with an interface circuit that can be connected on the in-house LAN. In this case, the CT image data can be fetched over the in-house LAN.
- the input unit 4has: as shown in FIG. 2 , a Next VBS switch SW 1 for use in directing update of a VBS image that is contained in a navigation screen and that represents each branch point of bronchi; an Auto Image Rotation switch SW 2 for use in directing automatic rotation of a VBS image so that the direction of the VBS image will correspond to the direction of a live image; a Manual Image Rotation switch SW 3 for use in directing rotation of a VBS image in units of a predetermined angle of rotation so that the direction of the VBS image will correspond to the direction of a live image; and a Freeze switch SW 4 for use in directing fetching of a still image of a live image.
- a Next VBS switch SW 1for use in directing update of a VBS image that is contained in a navigation screen and that represents each branch point of bronchi
- an Auto Image Rotation switch SW 2for use in directing automatic rotation of a VBS image so that the direction of the VBS image will correspond to the direction of a live image
- the bronchoscope navigation device 6is activated prior to observation or treatment performed using the bronchoscope 2 .
- the CT image data fetching unit 11fetches three-dimensional image data of a patient, which is constructed by a CT system.
- the fetched three-dimensional image datais stored in the CT image data storage unit 12 .
- the route determination unit 14displays the route determination screen 21 like the one shown in FIG. 4 on the monitor 5 .
- Patient informationis selected through a patient information tag screen 22 contained in the route determination screen 21 .
- an MPR imageis constructed from multiple different planar images, for example, three different multi-sectional images of a patient selected at step S 4 .
- the MPR image 23is contained in the route determination screen 21 .
- patient informationis selected through the patient information tag screen 22 by entering a patient identifier (ID), with which a patient is identified, using the setting information input unit 19 .
- IDpatient identifier
- step S 6the setting information input unit 19 is used to select a route determination tag 24 (see FIG. 4 ) in the route determination screen 21 . Consequently, a route determination tag screen 25 like the one shown in FIG. 5 appears in the route determination screen 21 . Route determination that will be described later is then executed in order to determine a route along which the bronchoscope 2 is navigated through bronchi in order to help insert the bronchoscope 2 .
- the VBS image construction unit 15constructs successive VBS images, which represent regions along the entire route, in units of a frame.
- the constructed VBS imagesare stored in the VBS image storage unit 16 .
- the bronchoscope navigation device 6 employed in the present embodimentcan navigate the bronchoscope 2 through bronchi so as to help insert the bronchoscope 2 according to, for example, two navigation methods to be described later.
- the methodsshall be implemented in mode 1 and mode 2 respectively.
- the setting information input unit 19is used to determine mode (navigation mode) for the route determination unit 14 .
- the information of the navigation modeis transmitted to the image processing unit 17 via the route determination unit 14 , and stored in both the image processing unit 17 and route determination unit 14 .
- a start point entry direction window 31 that prompts a user to enter a start point of a route like the one shown in FIG. 7is displayed in the route determination screen 21 at step S 11 .
- a start pointis defined in one of the tomographic images constituting the MPR image 23 by moving a cursor 30 in the route determination screen 21 . Once the start point is defined, the start point is indicated at corresponding positions in the other two tomographic images included in the MPR image 23 .
- An end point entry direction window 32 that prompts a user to enter an end point of the route like the one shown in FIG. 8is displayed in the route determination screen 21 .
- an end pointis defined in one of the tomographic images constituting the MPR image 23 by moving the cursor 30 in the route determination screen 21 . Once the end point is defined, the end point is indicated at corresponding positions in the other two tomographic images included in the MPR image 23 .
- the route determination unit 14searches a route that links the start point and end point and that runs through bronchi.
- the bronchihave complex paths.
- the route that links the start point and end point and that runs through bronchicannot therefore always be determined unconditionally.
- the route determination unit 14therefore searches a first candidate for the route that links the start point and end point and that runs through bronchi.
- the route determination unit 14then displays the route searched at step S 14 by superimposing the route on the MPR image 23 contained in the route determination screen 21 as shown in FIG. 9 . Moreover, a route finalization window 33 that prompts a user to finalize a route is displayed.
- the route finalization window 33comprises: a Finalize Route button 41 for use in directing finalization of a searched route; a Search Next Candidate button 42 for use in directing search of the next candidate for a route; a Re-determine Route button 43 for use in redefining a start point and an end point; and a Cancel button 44 for use in canceling route search.
- step S 15it is judged whether the Search Next Candidate button 42 is clicked. If the Search Next Candidate button 42 is clicked, the next candidate for a route is automatically searched at step S 16 . Control is then passed to step S 17 . If the Search Next Candidate button 42 is not clicked, control is passed to step S 18 . At step S 17 , it is judged whether the next candidate is found as a result of search. If the next candidate is unfound, the warning that the next candidate for a route is unfound is displayed, and control is returned to step S 13 . If the next candidate is found, control is returned to step S 14 .
- step S 19it is judged whether the Finalize Route button 41 is clicked. If the Finalize Route button 41 is not clicked, control is returned to step S 15 . If the Finalize Route button 41 is clicked, control is passed to step S 20 . At step S 20 , a route and positions of branch points on the route are determined, and control is returned to step S 7 in FIG. 6 .
- the order numbers of the branch points lying on the course to be steered in order to insert the bronchoscope from a start point 101 to an end point 102are appended to the VBS images representing the branch points.
- the VBS imagesare then stored in the VBS image storage unit 16 at step S 8 . Consequently, the VBS image data representing each branch point includes virtual image data that represents the branch portion, three-dimensional position information concerning the branch portion, and information of an order number assigned to the branch portion lying on the selected course to be steered in order to insert the bronchoscope.
- An image 100 constructed using a volume rendering technique as shown in FIG. 10may be used to designate the start point 101 and end point 102 for the purpose of route search.
- the navigation screen 51comprises: an endoscopic live image display area 52 in which a live image captured by the bronchoscope 2 is displayed; a VBS image display area 53 in which a VBS image is displayed; and a thumbnail VBS image area 54 in which images formed reducing VBS images representing branch points on a route are displayed as branch thumbnail VBS images.
- step S 21at which the bronchoscope 2 has not yet been inserted, no live image is displayed in the endoscopic live image display area 52 .
- a VBS image 53 a depicting the first branch point on the routeis displayed in the VBS image display area 53 .
- Thumbnail VBS images 54 ( a ) to 54 ( j ) representing all the branch pointsare displayed in the thumbnail VBS image area 54 .
- a marker 55is superimposed on the VBS image 53 a in order to indicate a lumen of a path that advances along the route.
- the frame of the thumbnail VBS image representing the same branch point as the VBS image 53 a displayed in the VBS image display area 53is displayed in boldface or in color so that the thumbnail VBS image can be discriminated from the others. An operator can easily discern which of the branch points is represented by a VBS image displayed in the VBS image display area 53 .
- the frame of the thumbnail VBS image 54 ( a )is displayed in boldface or in color.
- step S 22inserting the bronchoscope 2 into bronchi is started.
- a live image 52 a captured by the bronchoscope 2is displayed in the endoscopic live image display area 52 within the navigation screen 51 .
- the VBS image 53 a representing the first branch point on the routeis displayed in the VBS image display area 53
- the thumbnail VBS images 52 ( a ) to 54 ( j ) representing all the branch pointsare displayed in the thumbnail VBS image area 54 .
- the frame of the thumbnail VBS image 54 ( a )is displayed in boldface or in color.
- step S 24it is judged at step S 24 whether navigation mode is set to mode 1 in the image processing unit 17 that has received a live image.
- an operatorjudges whether or not to press the Freeze switch SW 4 . If the operator can discern the image of the lumen of the path indicated with the marker 55 on the live image 52 a by merely viewing the VBS image 53 a in the VBS image display area 53 and the live image 52 a , the operator keeps inserting the bronchoscope. If it is hard to discern the lumen of the path indicated with the marker 55 on the live image 52 a , the operator presses the Freeze switch SW 4 .
- step S 25it is judged whether the Freeze switch SW 4 is turned on. If the Freeze switch SW 4 is turned on, a still image acquired from the live image 52 a is fetched at step S 26 . The still image and the VBS image 53 a representing the first branch point are compared with each other in order to detect the degree of similarity between them according to a known image processing technique. If the Freeze switch SW 4 is not turned on, control is passed to step S 27 .
- step S 26If it is judged at step S 26 that the still image of the live image 52 a and the VBS image 53 a resemble each other at a degree of similarity exceeding the predetermined value, both the images are regarded to represent the same branch point.
- the operatorthen handles the Auto Image Rotation switch SW 2 or Manual Image Rotation switch SW 3 so as to rotate the VBS image 53 a so that the still image of the live image 52 a and the VBS image 53 a will be matched with each other. Thereafter, control is passed to step S 27 .
- the Auto Image Rotation switch SW 2when an operator presses the Auto Image Rotation switch SW 2 , the still image of the live image 52 a and the VBS image 53 a are automatically matched with each other through, for example, pattern recognition processing. Moreover, when the operator presses the Manual Image Rotation switch SW 3 , the VBS image 53 a is rotated by a predetermined angle. The operator repeatedly presses the switch SW 3 so as to match the live image 52 a and VBS image 53 a with each other.
- the operatorcan now discern the lumen of the path that is indicated with the marker 55 on the live image 52 a .
- the operatortherefore keeps inserting the bronchoscope according to the directive by the marker 55 .
- step S 27it is judged whether the operator has pressed the Next VBR switch SW 1 . If the Next VBR switch SW 1 is not pressed, control is returned to step S 25 . If the operator has pressed the Next VBR switch SW 1 , the VBS image 53 a corresponding to a thumbnail VBS image (b) representing the next (second) branch point is displayed in the VBS image display area 53 . At this time, the frame of the thumbnail VBS image 54 ( b ) representing the second branch point is displayed in boldface or in color. The frame of the thumbnail VBS image 54 ( a ) becomes the same as the frames of the other thumbnail VBS images.
- step S 25 to step S 29A concrete example of the process from step S 25 to step S 29 will be described in conjunction with the navigation screen 51 .
- the VBS image 53 a corresponding to a thumbnail VBS image 54 ( h ) representing the eighth branch pointis displayed in the VBS image display area 53 .
- the live image 52 a representing a region near the eighth branch pointis displayed in the endoscopic live image display area 52 .
- only the frame of the thumbnail VBS image 54 ( h ) representing the eighth branch pointis displayed in boldface or in color. This allows an operator to recognize that navigation is performed at the eighth branch point.
- step S 25if the operator presses the Freeze switch SW 4 (step S 25 ), a still image acquired from the live image is fetched. If the operator presses the Auto Image Rotation switch SW 2 , the VBS image 53 a is rotated as shown in FIG. 15 so that the still image of the live image 52 and the VBS image 53 a will be matched with each other (step S 26 ).
- step S 27the VBS image 53 a corresponding to a thumbnail VBS image 54 ( i ) representing the ninth branch point is displayed in the VBS image display area 53 . Only the frame of the thumbnail VBS image 54 ( i ) representing the ninth branch point is displayed in boldface or in color, whereby the operator recognizes that the next position of navigation is the ninth branch point (step S 28 ). Navigation is repeated in the same manner until the bronchoscope reaches the lesion (that is, the end point of navigation) (step S 29 ).
- a thumbnail VBS image representing a branch point at which navigation is expectedis searched based on a degree of insertion to which the insertion unit of a bronchoscope is inserted.
- the thumbnail VBS image 53 ais then displayed in the VBS image display area 53 , whereby navigation is enabled.
- navigation mode 2if an operator does not need navigation, the operator continues insertion of the bronchoscope 2 while monitoring a live image.
- the operatorpresses the Freeze switch SW 4 .
- Step S 41it is waited until the Freeze switch SW 4 is pressed. If the switch SW 4 is pressed, a reduced image 60 of a still image acquired with the press of the Freeze switch SW 4 is superimposed on the live image 52 a displayed in the endoscopic live image display area 52 .
- the Freeze switch SW 4when the Freeze switch SW 4 is pressed, the bronchoscope lies at the eighth branch point. Navigation has not been needed by this time.
- the Freeze switch SW 4is pressed for the first time.
- the VBS image 53 a displayed in the VBS image display area 53is a VBS image corresponding to a thumbnail VBS image 54 ( a ) representing the first branch point.
- the degree-of-insertion detection unit 3detects a degree of insertion to which the insertion unit of the bronchoscope 2 has been inserted into a patient body.
- the position of the distal end of the bronchoscope 2 and the position of a nearby branch pointare calculated based on the detected degree of insertion.
- VBS images representing regions started with a branch point immediately preceding the calculated branch point and ended with an immediately succeeding branch pointare displayed in the form of a motion picture in the VBS image display area 53 .
- the frame of only a thumbnail VBS image representing the calculated branch pointis displayed in boldface or in color.
- FIG. 19shows display of a motion picture composed of VBS images representing regions that start with the seventh branch point immediately preceding the eighth branch point, which is shown in FIG. 18 and serves as a reference point, and that end with the ninth branch point immediately succeeding the eighth branch point.
- FIG. 20shows the navigation screen 51 with the VBS image 53 a being displayed, which represents the seventh branch point and with which display of the motion picture is started, displayed in the VBS image display area 53 thereof.
- FIG. 21shows the navigation screen 51 with the VBS image 53 a being displayed, which represents the ninth branch point and with which the display of the motion picture is terminated, displayed in the VBS image display area 53 thereof.
- a motion picture domain bar 71that indicates a domain to be displayed in the form of a motion picture in the VBS image display area 53 is displayed below the thumbnail VBS images displayed in the thumbnail VBS image area 54 .
- FIG. 20 and FIG. 21show examples of display in which the motion picture domain bar 71 is displayed below the thumbnail VBS images that start with the thumbnail VBS image 54 ( g ) representing the seventh branch point and that end with the thumbnail VBS image 54 ( i ) representing the ninth branch point.
- the frame of only the thumbnail VBS image 54 ( h ) representing the calculated branch pointis displayed in boldface or in color.
- an operatorcan recognize at what branch point the distal end of the insertion unit is currently located.
- the operatorcan therefore advance or withdraw the inserted bronchoscope so as to find out a live image representing a nearby branch point.
- the live image 52 a representing the branch pointis displayed in the endoscopic live image display area 52 .
- step S 46Once the live image 52 a representing a position at which navigation is needed is thus displayed, it is waited at step S 46 until the Freeze switch SW 4 is turned on. If the switch SW 4 is turned on, a still image of the live image 52 a is fetched at step S 47 . The VBS image 53 a representing the calculated branch point is displayed in the VBS image display area 53 . At step S 48 , the still image of the live image 52 a and the VBS image 53 a representing the calculated branch point are compared with each other in order to detect the degree of similarity between the images according to the known image processing technique. A process similar to the one executed in mode 1 is then executed.
- the VBS image 53 a corresponding to the thumbnail VBS image 54 ( h ) representing the calculated branch pointis, as shown in FIG. 22 , displayed in the VBS image display area 53 .
- the frame of only the thumbnail VBS image 54 ( h ) representing the calculated branch pointis displayed in boldface or in color.
- step S 48similarly to the one in mode 1, if the degree of similarity is equal to or smaller than a predetermined value, it is judged that the bronchoscope has been inserted along a route other than a determined route (or the still image is compared with a VBS image representing a different branch point). A warning is then displayed (not shown).
- step S 48similarly to the one in mode 1, if it is judged that the degree of similarity exceeds the predetermined value and the still image of the live image 52 a and the VBS image 53 a resemble each other, both the images are regarded to represent the same branch point. An operator then handles the Auto Image Rotation switch SW 2 or Manual Image Rotation switch SW 3 so as to rotate the VBS image 53 a . The still image of the live image 52 a and the VBS image 53 a are thus matched with each other. Thereafter, control is passed to step S 49 .
- step S 49it is judged whether the bronchoscope has reached a lesion (that is, whether the bronchoscope has reached the end point of navigation). If the bronchoscope has reached the lesion, the process is terminated. If the bronchoscope has not reached the lesion, control is returned to step S 41 . The process from step S 41 to step S 49 will be repeated until the bronchoscope reaches the lesion.
- VBS images representing regions that start with a branch point immediately preceding a calculated branch point serving as a reference point and that end with an immediately succeeding branch pointare displayed in the form of a motion picture in the VBS image display area 53 .
- the present inventionis not limited to this mode.
- VBS images representing regions that start with a branch point of two points preceding a calculated branch point serving as a reference point and that end with a two point succeeding branch pointmay be displayed in the form of a motion picture in the VBS image display area 53 .
- thumbnail VBS images representing all branch pointsare displayed in the thumbnail VBS image area 54 .
- the frame of a thumbnail VBS image corresponding to the VBS image 53 awhich is compared with the live image 52 a , is displayed in boldface or in color. Consequently, which of branch points is represented by the live image 52 a can be easily discerned.
- the VBS image 53 arepresenting a branch point at which navigation is needed can be retrieved easily.
- the VBS images 53 a representing regions near a region represented by the live image 52 aare displayed in the form of a motion picture. Consequently, what branch point is represented by the live image 52 a can be easily discerned.
- the live image 52 a and VBS image 53 aare compared with each other, and the degree of similarity between the images is calculated according to an image processing technique. If the degree of similarity is equal to or smaller than a predetermined value, a warning is given. An operator suspends insertion at a branch point which the operator has misidentified. The operator can easily dispose the distal end of the bronchoscope 2 at an appropriate branch point.
- the frame of the thumbnail VBS image that corresponds to the VBS image 53 a and that is displayed in the thumbnail VBS image area 54is displayed in boldface or in color.
- the thumbnail VBS image corresponding to the VBS image 53 amay be enlarged, and the frame of the enlarged thumbnail VBS image may be displayed in boldface or in color.
- the number of branch pointsis nine, and the VBS image 53 a corresponding to the thumbnail VBS image representing the seventh branch point is displayed.
- thumbnail VBS images representing branch pointsare displayed in a row. If the number of branch points is large, thumbnail VBS images representing branch points may be, as shown in FIG. 24 , displayed in two or more rows. Moreover, as shown in FIG. 25 , if the frame of the thumbnail VBS image area 54 is structured such that thumbnail VBS images can be scrolled, a plurality of thumbnail VBS images can be scrolled and thus displayed.
- the marker 55is superimposed on the VBS image 53 a in the navigation screen 51 in order to indicate a lumen of a path extending along a route.
- the present inventionis not limited to this mode.
- the contour of the periphery of a lumen of a path extending along a route which is depicted in the VBS image 53 amay be displayed while being enhanced. Otherwise, the interior of a lumen of a path may be depicted in color.
- upward and downward directions(directions of gravity) depending on a patient's posture may be indicated in the VBS image 53 a within the navigation screen 51 .
- the upward and downward directionsmay also be, as shown in FIG. 28 , indicated in the live image 52 a.
- a marker 81may be superimposed on the VBS image 53 a in order to indicate the position of a tissue that is an object of biopsy and that is located at a destination of insertion. The distance from a current position to the position of the tissue that is an object of biopsy and the diameter of the tissue may be indicated in the VBS image 53 a .
- a tissue image 82that is patterned after the diameter or shape of a tissue may be displayed on the VBS image 53 a.
- an endoscopecan be reliably navigated to reach a target region using guide images representing actual branch positions.
- an endoscope device in accordance with the present inventionwill prove effective when used as: an endoscope device that navigates an endoscope so as to help insert the endoscope into an intracorporeal lumen; an endoscope device that navigates a bronchoscope so as to help insert the bronchoscope into an intracorporeal lumen such as bronchi; or an endoscope device for industrial use that navigates an endoscope so as to help insert the endoscope into a lumen that branches out at points in multiple stages.
- an endoscope device in accordance with the present inventionis the most suitable for reliably navigating the distal end of a bronchoscope.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- Optics & Photonics (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Pulmonology (AREA)
- Signal Processing (AREA)
- Otolaryngology (AREA)
- Physiology (AREA)
- Endoscopes (AREA)
- Eye Examination Apparatus (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Instruments For Viewing The Inside Of Hollow Bodies (AREA)
Abstract
Description
- The present invention relates to an endoscope device, or more particularly, an endoscope device capable of navigating an endoscope to help insert the endoscope into an intracorporeal lumen, for example, the bronchi.
- In recent years, diagnosis based on images has been widely adopted. For example, an X-ray computed tomography (CT) device is used to produce tomographic images of a subject so as to construct three-dimensional image data representing the interior of the subject. The three-dimensional image data is used to diagnose a lesion.
- As far as the CT system is concerned, while an X-ray irradiation and detection unit is continuously rotated, a subject is continuously moved in the direction of the body axis. Thus, helical scanning is performed on the three-dimensional field of the subject, and a three-dimensional image is produced based on tomographic images that represent successive sections of the subject in the three-dimensional field.
- One of such three-dimensional images is a three-dimensional image representing the bronchi in the lungs. The three-dimensional image of the bronchi is utilized in order to three-dimensionally grasp the position of an abnormality that is suspected to be, for example, a pulmonary carcinoma. In order to check the abnormality through biopsy, a bronchoscope is inserted in order to collect a sample of a tissue using a biopsy needle located in the distal section of the endoscope.
FIG. 31 shows abronchi 500. As far as an intracorporeal lumen that branches out in multiple stages is concerned, when the position of an abnormality is close to the terminal of a branch, it is hard to lead the distal end of an endoscope to a target region accurately for a short period of time. For example, Japanese Unexamined Patent Application Publication No. 2000-135215 has proposed a system for navigating a bronchoscope to a target region. Herein, a three-dimensional image of an intracorporeal lumen of a subject is constructed based on image data acquired from a three-dimensional field of the subject. The course to a target point along the lumen is determined using the three-dimensional image. Virtual endoscopic images representing regions of the lumen that lie along the course are constructed based on the image data, and displayed in order to navigate a bronchoscope.- However, the navigation to a target region to be performed in the system described in the Japanese Unexamined Patent Application Publication No. 2000-135215 includes display of a live endoscopic image produced by the bronchoscope. In addition, virtual endoscopic images representing branch points of bronchi are displayed for the purpose of guiding the bronchoscope into a destination of insertion. As mentioned above, the bronchi branch out in multiple stages. Moreover, images constructed with the bronchoscope located in the respective bronchi are images representing a plurality of branch destinations and resembling one another. When virtual endoscopic images are merely displayed, even if the virtual endoscopic images represent branch positions different from actual branch positions represented by a live endoscopic image, an operator may misidentify the virtual endoscopic images as images representing the correct branch positions. The misidentification may become a serious obstacle to navigation of the bronchoscope to a target region.
- The present invention attempts to break through the foregoing situation. An object of the present invention is to provide an endoscope device capable of reliably navigating an endoscope to a target region using guide images that represent actual branch positions.
- An endoscope device in accordance with the present invention comprises: three-dimensional image constructing means for constructing a three-dimensional image of an intracorporeal lumen of a subject on the basis of images representing a three-dimensional field of the subject; an endoscope that picks up images of the intracorporeal lumen of the subject; and navigation image constructing means that constructs a navigation image which includes the endoscopic image of the intracorporeal lumen of the subject that is produced by the endoscope and the three-dimensional image. While presenting the course of the endoscope to be steered in order to insert the endoscope into the intracorporeal lumen of the subject, the endoscope device helps observe or treat the subject. Herein, the navigation image constructing means constructs the navigation image adding reduced images of three-dimensional images, which represent all branch points at which the intracorporeal lumen of the subject branches out, added thereto. Owing to these constituent features, the endoscope can be reliably navigated to reach a target region using guide images representing the actual branch positions.
FIG. 1 shows the configuration of an endoscope device in accordance with an embodiment of the present invention;FIG. 2 shows the configuration of an input unit shown inFIG. 1 ;FIG. 3 is a flowchart describing the flow of constructing navigation data to be executed by a bronchoscope navigation device shown inFIG. 1 ;FIG. 4 is a first diagram showing a route determination screen in which the execution of the process described inFIG. 3 is performed;FIG. 5 is a second diagram showing the route determination screen in which the execution of the process described inFIG. 3 is performed;FIG. 6 is a flowchart describing the flow of determining a route;FIG. 7 is a first diagram showing a route determination screen in which the execution of the process described inFIG. 6 is performed;FIG. 8 is a second diagram showing the route determination screen in which the execution of the process described inFIG. 6 is performed;FIG. 9 is a third diagram showing the route determination screen in which the execution of the process described inFIG. 6 is performed;FIG. 10 is an explanatory diagram concerning a variant of route search to be executed by the bronchoscope navigation device shown inFIG. 1 ;FIG. 11 is a first flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown inFIG. 1 ;FIG. 12 is a first diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed;FIG. 13 is a second diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed;FIG. 14 is a third diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed;FIG. 15 is a fourth diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed;FIG. 16 is a fifth diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed;FIG. 17 is a second flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown inFIG. 1 ;FIG. 18 is a first diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed;FIG. 19 is an explanatory diagram concerning a motion picture to be displayed by executing the process described inFIG. 17 ;FIG. 20 is a second diagram showing the navigation screen in which the execution of the process described inFIG. 17 is performed;FIG. 21 is a third diagram showing the navigation screen in which the execution of the process described inFIG. 17 is performed;FIG. 22 is a third diagram showing the navigation screen in which the execution of the process described inFIG. 17 is performed;FIG. 23 shows a first variant of the navigation screen shown inFIG. 14 ;FIG. 24 is a second variant of the navigation screen shown inFIG. 14 ;FIG. 25 is a third variant of the navigation screen shown inFIG. 14 ;FIG. 26 is a fourth variant of the navigation screen shown inFIG. 14 ;FIG. 27 is a fifth variant of the navigation screen shown inFIG. 14 ;FIG. 28 is a sixth variant of the navigation screen shown inFIG. 14 ;FIG. 29 is a seventh variant of the navigation screen shown inFIG. 14 ;FIG. 30 is an eighth variant of the navigation screen shown inFIG. 14 ; andFIG. 31 shows the structure of bronchi.- Referring to the drawings, an embodiment of the present invention will be described below.
FIG. 1 toFIG. 30 are concerned with an embodiment of the present invention.FIG. 1 shows the configuration of an endoscope device.FIG. 2 shows the configuration of an input unit shown inFIG. 1 .FIG. 3 is a flowchart describing the flow of constructing navigation data to be performed by a bronchoscope navigation device shown inFIG. 1 .FIG. 4 is a first diagram showing a route determination screen in which the execution of the process described inFIG. 3 is performed.FIG. 5 is a second diagram showing the route determination screen in which the execution of the process described inFIG. 3 is performed.FIG. 6 is a flowchart describing the flow of route determination to be executed at a step in the process described inFIG. 3 .FIG. 7 is a first diagram showing a route determination screen in which the execution of the process described inFIG. 6 is performed.FIG. 8 is a second diagram showing the route determination screen in which the execution of the process described inFIG. 6 is performed.FIG. 9 is a third diagram showing the route determination screen in which the execution of the process described inFIG. 6 is performed.FIG. 10 is an explanatory diagram concerning a variant of route search to be executed by the bronchoscope navigation device shown inFIG. 1 .FIG. 11 is a first flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown inFIG. 1 .FIG. 12 is a first diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 13 is a second diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 14 is a third diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 15 is a fourth diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 16 is a fifth diagram showing a navigation screen in which the execution of the process described inFIG. 11 is performed.FIG. 17 is a second flowchart describing the flow of navigation to be executed by the bronchoscope navigation device shown inFIG. 1 .FIG. 18 is a first diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed.FIG. 19 is an explanatory diagram concerning a motion picture to be displayed by executing the process described inFIG. 17 .FIG. 20 is a second diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed.FIG. 21 is a third diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed.FIG. 22 is a third diagram showing a navigation screen in which the execution of the process described inFIG. 17 is performed.FIG. 23 shows a first variant of the navigation screen shown inFIG. 14 .FIG. 24 shows a second variant of the navigation screen shown inFIG. 14 .FIG. 25 is a third variant of the navigation screen shown inFIG. 14 .FIG. 26 shows a fourth variant of the navigation screen shown inFIG. 14 .FIG. 27 is a fifth variant of the navigation screen shown inFIG. 14 .FIG. 28 is a sixth variant of the navigation screen shown inFIG. 14 .FIG. 29 is a seventh variant of the navigation screen shown inFIG. 14 .FIG. 30 is an eighth variant of the navigation screen shown inFIG. 14 .- As shown in
FIG. 1 , anendoscope device 1 in accordance with the present embodiment comprises: abronchoscope 2 that is inserted into the patient's bronchi in order to pick up images of the interiors of the bronchi or to biopsy a tissue of a lesion at the terminal of a bronchus; a degree-of-insertion detection unit 3 that detects a degree of insertion of the insertion unit of thebronchoscope 2 into the patient's body and that includes, for example, an encoder; aninput unit 4 formed with a flexible sheet that realizes a plurality of switches in the form of a film and that is disposed on a proximal end of thebronchoscope 2; and abronchoscope navigation device 6 that constructs virtual endoscopic images (hereinafter, VBS images) representing the interiors of bronchi on the basis of CT image data, synthesizes an endoscopic image (hereinafter, a live image), which is produced by thebronchoscope 2, with the VBS images according to an image pickup signal sent from thebronchoscope 2, a detection signal sent from the degree-of-insertion detection unit 3, and an input signal sent from theinput unit 4, displays the resultant synthetic image on amonitor 5, and navigates thebronchoscope 2 through the bronchi. - Incidentally, the
input unit 4 is formed on the proximal end of thebronchoscope 2. Alternatively, theinput unit 4 may be formed with a footswitch. - The bronchoscope navigation device6 comprises: a CT image data fetching unit11 that fetches three-dimensional image data, which is constructed in a known CT system that is not shown and that produces X-ray tomographic images of a patient, via a transportable storage medium, for example, a magneto-optical (MO) disk or a digital versatile disk (DVD); a CT image data storage unit12 in which three-dimensional image data fetched by the CT image data fetching unit11 is stored; an MPR image construction unit13 that constructs an MPR image on the basis of the three-dimensional image data stored in the CT image data storage unit12; a route determination unit14 that constructs a route determination screen that will be described later and that contains an MPR image constructed by the MPR image constructing unit, and determines a navigation route (hereinafter, simply, a route) along which the bronchoscope2 is navigated through bronchi; a VBS image construction unit15 that constructs VBS images, which represent successive regions along the route determined by the route determining unit14, in units of a frame on the basis of the three-dimensional image data saved in the CT image data storage unit12; a VBS image storage unit16 in which VBS images constructed by the VBS image constructing unit15 are stored; an image processing unit17 that receives an image pickup signal sent from the bronchoscope2, a detection signal sent from the degree-of-insertion detection unit3, and an input signal sent from the input unit4, and constructs a navigation screen that will be described later and that comprises a live image, a VBS image, and a plurality of thumbnail VBS images; an image display control unit18 that displays the route determination screen, which is constructed by the route determination unit14, and the navigation screen, which is constructed by the image processing unit17, on the monitor5; and a setting information input unit19 including a keyboard and a pointing device to be used to enter setting information in the route determination unit14.
- Incidentally, the CT image
data storage unit 12 and VBSimage storage unit 16 may be realized with one hard disk, and the MPRimage construction unit 13,route determination unit 14, VBSimage construction unit 15, andimage processing unit 17 may be realized with one arithmetic circuit. Moreover, the CT imagedata fetching unit 11 fetches CT image data from a transportable storage medium such as a MO disk or a DVD. Alternatively, when a CT system or an in-house server in which CT image data is preserved is connected on a in-house LAN, the CT imagedata fetching unit 11 may be realized with an interface circuit that can be connected on the in-house LAN. In this case, the CT image data can be fetched over the in-house LAN. - The
input unit 4 has: as shown inFIG. 2 , a Next VBS switch SW1 for use in directing update of a VBS image that is contained in a navigation screen and that represents each branch point of bronchi; an Auto Image Rotation switch SW2 for use in directing automatic rotation of a VBS image so that the direction of the VBS image will correspond to the direction of a live image; a Manual Image Rotation switch SW3 for use in directing rotation of a VBS image in units of a predetermined angle of rotation so that the direction of the VBS image will correspond to the direction of a live image; and a Freeze switch SW4 for use in directing fetching of a still image of a live image. - The operation of the present embodiment having the foregoing components will be described below.
- As shown in
FIG. 3 , prior to observation or treatment performed using thebronchoscope 2, thebronchoscope navigation device 6 is activated. In thebronchoscope navigation device 6, at step S1, the CT imagedata fetching unit 11 fetches three-dimensional image data of a patient, which is constructed by a CT system. At step S2, the fetched three-dimensional image data is stored in the CT imagedata storage unit 12. - At step S3, the
route determination unit 14 displays theroute determination screen 21 like the one shown inFIG. 4 on themonitor 5. Patient information is selected through a patientinformation tag screen 22 contained in theroute determination screen 21. With the selection, an MPR image is constructed from multiple different planar images, for example, three different multi-sectional images of a patient selected at step S4. At step S5, theMPR image 23 is contained in theroute determination screen 21. - Incidentally, patient information is selected through the patient
information tag screen 22 by entering a patient identifier (ID), with which a patient is identified, using the settinginformation input unit 19. - Next, at step S6, the setting
information input unit 19 is used to select a route determination tag24 (seeFIG. 4 ) in theroute determination screen 21. Consequently, a routedetermination tag screen 25 like the one shown inFIG. 5 appears in theroute determination screen 21. Route determination that will be described later is then executed in order to determine a route along which thebronchoscope 2 is navigated through bronchi in order to help insert thebronchoscope 2. - After a route along which the bronchoscope is navigated in order to help insert the bronchoscope is determined, the VBS
image construction unit 15 constructs successive VBS images, which represent regions along the entire route, in units of a frame. At step S8, the constructed VBS images are stored in the VBSimage storage unit 16. - The
bronchoscope navigation device 6 employed in the present embodiment can navigate thebronchoscope 2 through bronchi so as to help insert thebronchoscope 2 according to, for example, two navigation methods to be described later. The methods shall be implemented inmode 1 andmode 2 respectively. At step S9, the settinginformation input unit 19 is used to determine mode (navigation mode) for theroute determination unit 14. The information of the navigation mode is transmitted to theimage processing unit 17 via theroute determination unit 14, and stored in both theimage processing unit 17 androute determination unit 14. - By executing step S1 to step S9, preparations are made for navigation to be performed by the
bronchoscope navigation device 6 for observation or treatment by thebronchoscope 2. - Now, route determination to be executed at step S6 will be described in conjunction with
FIG. 6 . - As shown in
FIG. 6 , during the route determination of step S6, the settinginformation input unit 19 is handled in order to click a Route Search button in the routesetting tag screen 25 shown inFIG. 5 . A start pointentry direction window 31 that prompts a user to enter a start point of a route like the one shown inFIG. 7 is displayed in theroute determination screen 21 at step S11. A start point is defined in one of the tomographic images constituting theMPR image 23 by moving acursor 30 in theroute determination screen 21. Once the start point is defined, the start point is indicated at corresponding positions in the other two tomographic images included in theMPR image 23. An end pointentry direction window 32 that prompts a user to enter an end point of the route like the one shown inFIG. 8 is displayed in theroute determination screen 21. - At step S12, similarly to the definition of the start point, an end point is defined in one of the tomographic images constituting the
MPR image 23 by moving thecursor 30 in theroute determination screen 21. Once the end point is defined, the end point is indicated at corresponding positions in the other two tomographic images included in theMPR image 23. - After the start point and end point are defined, the
route determination unit 14 searches a route that links the start point and end point and that runs through bronchi. The bronchi have complex paths. The route that links the start point and end point and that runs through bronchi cannot therefore always be determined unconditionally. At step S13, theroute determination unit 14 therefore searches a first candidate for the route that links the start point and end point and that runs through bronchi. - The
route determination unit 14 then displays the route searched at step S14 by superimposing the route on theMPR image 23 contained in theroute determination screen 21 as shown inFIG. 9 . Moreover, aroute finalization window 33 that prompts a user to finalize a route is displayed. - The
route finalization window 33 comprises: a FinalizeRoute button 41 for use in directing finalization of a searched route; a SearchNext Candidate button 42 for use in directing search of the next candidate for a route; aRe-determine Route button 43 for use in redefining a start point and an end point; and a Cancelbutton 44 for use in canceling route search. - At step S15, it is judged whether the Search
Next Candidate button 42 is clicked. If the SearchNext Candidate button 42 is clicked, the next candidate for a route is automatically searched at step S16. Control is then passed to step S17. If the SearchNext Candidate button 42 is not clicked, control is passed to step S18. At step S17, it is judged whether the next candidate is found as a result of search. If the next candidate is unfound, the warning that the next candidate for a route is unfound is displayed, and control is returned to step S13. If the next candidate is found, control is returned to step S14. - At step S18, it is judged whether the
Re-determine Route button 43 is clicked. If theRe-determine Route button 43 is clicked, control is returned to step S11. If theRe-determine Route button 43 is not clicked, control is passed to step S19. - At step S19, it is judged whether the Finalize
Route button 41 is clicked. If the FinalizeRoute button 41 is not clicked, control is returned to step S15. If the FinalizeRoute button 41 is clicked, control is passed to step S20. At step S20, a route and positions of branch points on the route are determined, and control is returned to step S7 inFIG. 6 . - After the three-dimensional position information of the branch points on the route is determined at step S20, the order numbers of the branch points lying on the course to be steered in order to insert the bronchoscope from a
start point 101 to anend point 102 are appended to the VBS images representing the branch points. The VBS images are then stored in the VBSimage storage unit 16 at step S8. Consequently, the VBS image data representing each branch point includes virtual image data that represents the branch portion, three-dimensional position information concerning the branch portion, and information of an order number assigned to the branch portion lying on the selected course to be steered in order to insert the bronchoscope. - Incidentally, route search to be performed using an MPR image has been described. An
image 100 constructed using a volume rendering technique as shown inFIG. 10 may be used to designate thestart point 101 andend point 102 for the purpose of route search. - A description will be made of navigation of the
bronchoscope 2 which thebronchoscope navigation device 6 performs in order to help insert thebronchoscope 2 for observation or treatment after a route is determined as mentioned above. Incidentally, the description will be made by taking for instance a case where the number of branch points on a route is ten. - As described in
FIG. 11 , when thebronchoscope navigation device 6 starts navigation, anavigation screen 51 like the one shown inFIG. 12 is displayed on themonitor 5 at step S21. - The
navigation screen 51 comprises: an endoscopic liveimage display area 52 in which a live image captured by thebronchoscope 2 is displayed; a VBSimage display area 53 in which a VBS image is displayed; and a thumbnailVBS image area 54 in which images formed reducing VBS images representing branch points on a route are displayed as branch thumbnail VBS images. At step S21 at which thebronchoscope 2 has not yet been inserted, no live image is displayed in the endoscopic liveimage display area 52. However, aVBS image 53adepicting the first branch point on the route is displayed in the VBSimage display area 53. Thumbnail VBS images54(a) to54(j) representing all the branch points are displayed in the thumbnailVBS image area 54. - A
marker 55 is superimposed on theVBS image 53ain order to indicate a lumen of a path that advances along the route. Moreover, the frame of the thumbnail VBS image representing the same branch point as theVBS image 53adisplayed in the VBSimage display area 53 is displayed in boldface or in color so that the thumbnail VBS image can be discriminated from the others. An operator can easily discern which of the branch points is represented by a VBS image displayed in the VBSimage display area 53. In the stage of step S21, the frame of the thumbnail VBS image54(a) is displayed in boldface or in color. - At step S22, inserting the
bronchoscope 2 into bronchi is started. As shown inFIG. 13 , at step S23, alive image 52acaptured by thebronchoscope 2 is displayed in the endoscopic liveimage display area 52 within thenavigation screen 51. Even inFIG. 13 , theVBS image 53arepresenting the first branch point on the route is displayed in the VBSimage display area 53, and the thumbnail VBS images52(a) to54(j) representing all the branch points are displayed in the thumbnailVBS image area 54. The frame of the thumbnail VBS image54(a) is displayed in boldface or in color. - Next, it is judged at step S24 whether navigation mode is set to
mode 1 in theimage processing unit 17 that has received a live image. - To begin with, navigation to be performed by the
bronchoscope navigation device 6 will be described on the assumption that the navigation mode is set tomode 1. - First, when the distal end of the
bronchoscope 2 reaches a first branch point, an operator judges whether or not to press the Freeze switch SW4. If the operator can discern the image of the lumen of the path indicated with themarker 55 on thelive image 52aby merely viewing theVBS image 53ain the VBSimage display area 53 and thelive image 52a, the operator keeps inserting the bronchoscope. If it is hard to discern the lumen of the path indicated with themarker 55 on thelive image 52a, the operator presses the Freeze switch SW4. - At step S25, it is judged whether the Freeze switch SW4 is turned on. If the Freeze switch SW4 is turned on, a still image acquired from the
live image 52ais fetched at step S26. The still image and theVBS image 53arepresenting the first branch point are compared with each other in order to detect the degree of similarity between them according to a known image processing technique. If the Freeze switch SW4 is not turned on, control is passed to step S27. - If it is found at step S26 that the degree of similarity is equal to or smaller than a predetermined value, it is judged that the bronchoscope has been inserted along a route other than the determined route (or, the still image is compared with a VBS image representing a different branch point). Consequently, a warning is displayed (not shown).
- If it is judged at step S26 that the still image of the
live image 52aand theVBS image 53aresemble each other at a degree of similarity exceeding the predetermined value, both the images are regarded to represent the same branch point. The operator then handles the Auto Image Rotation switch SW2 or Manual Image Rotation switch SW3 so as to rotate theVBS image 53aso that the still image of thelive image 52aand theVBS image 53awill be matched with each other. Thereafter, control is passed to step S27. - Incidentally, when an operator presses the Auto Image Rotation switch SW2, the still image of the
live image 52aand theVBS image 53aare automatically matched with each other through, for example, pattern recognition processing. Moreover, when the operator presses the Manual Image Rotation switch SW3, theVBS image 53ais rotated by a predetermined angle. The operator repeatedly presses the switch SW3 so as to match thelive image 52aandVBS image 53awith each other. - After the still image of the
live image 52aand theVBS image 53aare thus matched with each other, the operator can now discern the lumen of the path that is indicated with themarker 55 on thelive image 52a. The operator therefore keeps inserting the bronchoscope according to the directive by themarker 55. - At step S27, it is judged whether the operator has pressed the Next VBR switch SW1. If the Next VBR switch SW1 is not pressed, control is returned to step S25. If the operator has pressed the Next VBR switch SW1, the
VBS image 53acorresponding to a thumbnail VBS image (b) representing the next (second) branch point is displayed in the VBSimage display area 53. At this time, the frame of the thumbnail VBS image54(b) representing the second branch point is displayed in boldface or in color. The frame of the thumbnail VBS image54(a) becomes the same as the frames of the other thumbnail VBS images. - At step S29, it is judged whether the bronchoscope has reached a lesion (that is, reached the end point of navigation). If the bronchoscope has reached the lesion, the process is terminated. If the bronchoscope has not reached the lesion, control is returned to step S25. The process from step S25 to step S29 is repeated until the bronchoscope reaches the lesion.
- A concrete example of the process from step S25 to step S29 will be described in conjunction with the
navigation screen 51. As shown inFIG. 14 , assuming that the bronchoscope has been navigated to reach the eighth branch point, theVBS image 53acorresponding to a thumbnail VBS image54(h) representing the eighth branch point is displayed in the VBSimage display area 53. Thelive image 52arepresenting a region near the eighth branch point is displayed in the endoscopic liveimage display area 52. At this time, only the frame of the thumbnail VBS image54(h) representing the eighth branch point is displayed in boldface or in color. This allows an operator to recognize that navigation is performed at the eighth branch point. - In this state, if the operator presses the Freeze switch SW4 (step S25), a still image acquired from the live image is fetched. If the operator presses the Auto Image Rotation switch SW2, the
VBS image 53ais rotated as shown inFIG. 15 so that the still image of thelive image 52 and theVBS image 53awill be matched with each other (step S26). - Thereafter, after the operator recognizes the lumen of the path that is a destination of insertion, if the operator presses the Next VBS switch SW1 (step S27), the
VBS image 53acorresponding to a thumbnail VBS image54(i) representing the ninth branch point is displayed in the VBSimage display area 53. Only the frame of the thumbnail VBS image54(i) representing the ninth branch point is displayed in boldface or in color, whereby the operator recognizes that the next position of navigation is the ninth branch point (step S28). Navigation is repeated in the same manner until the bronchoscope reaches the lesion (that is, the end point of navigation) (step S29). - Next, a description will be made of navigation to be performed by the
bronchoscope navigation device 6 in a case where the navigation mode is set tomode 2. - If it is judged at step S24 described in
FIG. 11 that the navigation mode is not set tomode 1 in theimage processing unit 17 having received a live image but set tomode 2, control is passed to step S41 inFIG. 17 . - The
navigation mode 2 is mode to be utilized by an operator who has the expertise in inserting a bronchoscope into bronchi. When an operator expects navigation but has a thorough knowledge of lumens of paths that are destinations inserted at almost all branch points, the operator designates themode 2 instead of themode 1 because he/she does not need the navigation to a predetermined position at each branch point. - However, in
mode 1, every time the bronchoscope reaches a branch point, the Next VBS switch SW1 is pressed. The position of a branch point represented by a live image can therefore be matched with the position thereof represented by a VBS image. However, assume that navigation is not needed at each branch point until a bronchoscope reaches a predetermined position, insertion is continued, and navigation is expected at a certain branch point. In this case, it is hard to find a thumbnail VBS image representing the branch point at which navigation is expected. - In the
navigation mode 2, a thumbnail VBS image representing a branch point at which navigation is expected is searched based on a degree of insertion to which the insertion unit of a bronchoscope is inserted. Thethumbnail VBS image 53ais then displayed in the VBSimage display area 53, whereby navigation is enabled. - To be more specific, in
navigation mode 2, if an operator does not need navigation, the operator continues insertion of thebronchoscope 2 while monitoring a live image. When the distal end of thebronchoscope 2 reaches a branch point at which the operator needs navigation, the operator presses the Freeze switch SW4. - Navigation to be performed by the
bronchoscope navigation device 6 inmode 2 is described inFIG. 17 . Namely, at step S41, it is waited until the Freeze switch SW4 is pressed. If the switch SW4 is pressed, a reducedimage 60 of a still image acquired with the press of the Freeze switch SW4 is superimposed on thelive image 52adisplayed in the endoscopic liveimage display area 52. - Referring to
FIG. 18 , when the Freeze switch SW4 is pressed, the bronchoscope lies at the eighth branch point. Navigation has not been needed by this time. The Freeze switch SW4 is pressed for the first time. TheVBS image 53adisplayed in the VBSimage display area 53 is a VBS image corresponding to a thumbnail VBS image54(a) representing the first branch point. - At step S43, the degree-of-
insertion detection unit 3 detects a degree of insertion to which the insertion unit of thebronchoscope 2 has been inserted into a patient body. At step S44, the position of the distal end of thebronchoscope 2 and the position of a nearby branch point are calculated based on the detected degree of insertion. - At step S45, VBS images representing regions started with a branch point immediately preceding the calculated branch point and ended with an immediately succeeding branch point are displayed in the form of a motion picture in the VBS
image display area 53. Moreover, the frame of only a thumbnail VBS image representing the calculated branch point is displayed in boldface or in color. FIG. 19 shows display of a motion picture composed of VBS images representing regions that start with the seventh branch point immediately preceding the eighth branch point, which is shown inFIG. 18 and serves as a reference point, and that end with the ninth branch point immediately succeeding the eighth branch point.FIG. 20 shows thenavigation screen 51 with theVBS image 53abeing displayed, which represents the seventh branch point and with which display of the motion picture is started, displayed in the VBSimage display area 53 thereof.FIG. 21 shows thenavigation screen 51 with theVBS image 53abeing displayed, which represents the ninth branch point and with which the display of the motion picture is terminated, displayed in the VBSimage display area 53 thereof.- As shown in
FIG. 20 andFIG. 21 , a motion picture domain bar71 that indicates a domain to be displayed in the form of a motion picture in the VBSimage display area 53 is displayed below the thumbnail VBS images displayed in the thumbnailVBS image area 54.FIG. 20 andFIG. 21 show examples of display in which the motion picture domain bar71 is displayed below the thumbnail VBS images that start with the thumbnail VBS image54(g) representing the seventh branch point and that end with the thumbnail VBS image54(i) representing the ninth branch point. At this time, the frame of only the thumbnail VBS image54(h) representing the calculated branch point is displayed in boldface or in color. - Consequently, an operator can recognize at what branch point the distal end of the insertion unit is currently located. The operator can therefore advance or withdraw the inserted bronchoscope so as to find out a live image representing a nearby branch point. Eventually, the
live image 52arepresenting the branch point is displayed in the endoscopic liveimage display area 52. - Once the
live image 52arepresenting a position at which navigation is needed is thus displayed, it is waited at step S46 until the Freeze switch SW4 is turned on. If the switch SW4 is turned on, a still image of thelive image 52ais fetched at step S47. TheVBS image 53arepresenting the calculated branch point is displayed in the VBSimage display area 53. At step S48, the still image of thelive image 52aand theVBS image 53arepresenting the calculated branch point are compared with each other in order to detect the degree of similarity between the images according to the known image processing technique. A process similar to the one executed inmode 1 is then executed. - In the state shown in
FIG. 21 , if the Freeze switch SW4 is turned on, theVBS image 53acorresponding to the thumbnail VBS image54(h) representing the calculated branch point is, as shown inFIG. 22 , displayed in the VBSimage display area 53. Moreover, the frame of only the thumbnail VBS image54(h) representing the calculated branch point is displayed in boldface or in color. - At step S48, similarly to the one in
mode 1, if the degree of similarity is equal to or smaller than a predetermined value, it is judged that the bronchoscope has been inserted along a route other than a determined route (or the still image is compared with a VBS image representing a different branch point). A warning is then displayed (not shown). - Moreover, at step S48, similarly to the one in
mode 1, if it is judged that the degree of similarity exceeds the predetermined value and the still image of thelive image 52aand theVBS image 53aresemble each other, both the images are regarded to represent the same branch point. An operator then handles the Auto Image Rotation switch SW2 or Manual Image Rotation switch SW3 so as to rotate theVBS image 53a. The still image of thelive image 52aand theVBS image 53aare thus matched with each other. Thereafter, control is passed to step S49. - Once the still image of the
live image 52aand theVBS image 53aare matched with each other, an operator can easily discern a lumen of a path that is indicated with themarker 55 on thelive image 52a. The operator continues insertion according to the directive given with themarker 55. - At step S49, it is judged whether the bronchoscope has reached a lesion (that is, whether the bronchoscope has reached the end point of navigation). If the bronchoscope has reached the lesion, the process is terminated. If the bronchoscope has not reached the lesion, control is returned to step S41. The process from step S41 to step S49 will be repeated until the bronchoscope reaches the lesion.
- At step S45, VBS images representing regions that start with a branch point immediately preceding a calculated branch point serving as a reference point and that end with an immediately succeeding branch point are displayed in the form of a motion picture in the VBS
image display area 53. The present invention is not limited to this mode. Alternatively, VBS images representing regions that start with a branch point of two points preceding a calculated branch point serving as a reference point and that end with a two point succeeding branch point may be displayed in the form of a motion picture in the VBSimage display area 53. - According to the present embodiment, when navigation is performed in
mode 1, thumbnail VBS images representing all branch points are displayed in the thumbnailVBS image area 54. Moreover, the frame of a thumbnail VBS image corresponding to theVBS image 53a, which is compared with thelive image 52a, is displayed in boldface or in color. Consequently, which of branch points is represented by thelive image 52acan be easily discerned. - Moreover, when navigation is performed in
mode 2, theVBS image 53arepresenting a branch point at which navigation is needed can be retrieved easily. TheVBS images 53arepresenting regions near a region represented by thelive image 52aare displayed in the form of a motion picture. Consequently, what branch point is represented by thelive image 52acan be easily discerned. - Furthermore, when navigation is performed in
mode live image 52aandVBS image 53aare compared with each other, and the degree of similarity between the images is calculated according to an image processing technique. If the degree of similarity is equal to or smaller than a predetermined value, a warning is given. An operator suspends insertion at a branch point which the operator has misidentified. The operator can easily dispose the distal end of thebronchoscope 2 at an appropriate branch point. - Incidentally, in the
navigation screen 51, the frame of the thumbnail VBS image that corresponds to theVBS image 53aand that is displayed in the thumbnailVBS image area 54 is displayed in boldface or in color. As shown inFIG. 23 , the thumbnail VBS image corresponding to theVBS image 53amay be enlarged, and the frame of the enlarged thumbnail VBS image may be displayed in boldface or in color. Referring toFIG. 23 , the number of branch points is nine, and theVBS image 53acorresponding to the thumbnail VBS image representing the seventh branch point is displayed. - In the
navigation screen 51, thumbnail VBS images representing branch points are displayed in a row. If the number of branch points is large, thumbnail VBS images representing branch points may be, as shown inFIG. 24 , displayed in two or more rows. Moreover, as shown inFIG. 25 , if the frame of the thumbnailVBS image area 54 is structured such that thumbnail VBS images can be scrolled, a plurality of thumbnail VBS images can be scrolled and thus displayed. - Furthermore, the
marker 55 is superimposed on theVBS image 53ain thenavigation screen 51 in order to indicate a lumen of a path extending along a route. The present invention is not limited to this mode. For example, as shown inFIG. 26 , the contour of the periphery of a lumen of a path extending along a route which is depicted in theVBS image 53amay be displayed while being enhanced. Otherwise, the interior of a lumen of a path may be depicted in color. - Furthermore, as shown in
FIG. 27 , upward and downward directions (directions of gravity) depending on a patient's posture may be indicated in theVBS image 53awithin thenavigation screen 51. The upward and downward directions may also be, as shown inFIG. 28 , indicated in thelive image 52a. - Moreover, as shown in
FIG. 29 , amarker 81 may be superimposed on theVBS image 53ain order to indicate the position of a tissue that is an object of biopsy and that is located at a destination of insertion. The distance from a current position to the position of the tissue that is an object of biopsy and the diameter of the tissue may be indicated in theVBS image 53a. Instead of themarker 81, atissue image 82 that is patterned after the diameter or shape of a tissue may be displayed on theVBS image 53a. - Therefore, according to the present embodiment, an endoscope can be reliably navigated to reach a target region using guide images representing actual branch positions.
- The present invention is not limited to the foregoing embodiment. Various modifications and variations can be made within the gist of the present invention.
- As mentioned above, an endoscope device in accordance with the present invention will prove effective when used as: an endoscope device that navigates an endoscope so as to help insert the endoscope into an intracorporeal lumen; an endoscope device that navigates a bronchoscope so as to help insert the bronchoscope into an intracorporeal lumen such as bronchi; or an endoscope device for industrial use that navigates an endoscope so as to help insert the endoscope into a lumen that branches out at points in multiple stages. In particular, when a target region is located at a terminal of an intracorporeal lumen that branches out at points in multiple stages, such as, bronchi, an endoscope device in accordance with the present invention is the most suitable for reliably navigating the distal end of a bronchoscope.
Claims (27)
1. An endoscope device in accordance with the present invention comprising:
three-dimensional image constructing means for constructing a three-dimensional image of an intracorporeal lumen of a subject on the basis of image data acquired from a three-dimensional field of the subject;
an endoscope for imaging the intracorporeal lumen of the subject; and
navigation image constructing means for constructing a navigation image, which includes an endoscopic image that is produced by the endoscope and that represents the intracorporeal lumen of the subject, and the three-dimensional image, so that while presenting the course of the endoscope to be steered in order to insert the endoscope into the intracorporeal lumen of the subject, the endoscope device can help observe or treat the interior of the subject, wherein:
the navigation image constructing means constructs the navigation image having reduced images of three-dimensional images, which represent all branch points at which the intracorporeal lumen of the subject branches out, added thereto.
2. The endoscope device according toclaim 1 , wherein the navigation image constructing means comprises: route designating means that designates and calculates a route, which starts with a point at which the endoscope is inserted and ends with a point at which the endoscope reaches a target region of the subject, so that the course of the endoscope to be steered in order to insert the endoscope can be presented; and an image processing unit that constructs reduced images of three-dimensional images, which represent all the branch points on the determined route at which the intracorporeal lumen branches out, according to the result of calculation performed by the route designating means, and that constructs the navigation image having the reduced images added thereto.
3. The endoscopic imaging system according toclaim 2 , further comprising image display control means that displays the navigation image, which is constructed by the image processing unit included in the navigation image constructing means, on a monitor.
4. The endoscope device according toclaim 2 , wherein the navigation image constructed by the image processing unit includes the endoscopic image, the three-dimensional image, and the reduced images of three-dimensional images that represent all the branch points on the route determined by the route designating means at which the lumen branches out.
5. The endoscope device according toclaim 2 , wherein the route designating means can designate and calculate a plurality of routes that starts with the point at which the endoscope is inserted and ends with the point at which the endoscope reaches the target region.
6. The endoscope device according toclaim 2 , wherein the route designating means designates the route through a display screen image of the three-dimensional image that represents the intracorporeal lumen of the subject and that is constructed by the three-dimensional image constructing means.
7. The endoscope device according toclaim 2 , wherein: the navigation image constructing means includes multi-screen reconstructed image constructing means that constructs a multi-screen reconstructed image, which represents the intracorporeal lumen of the subject, on the basis of the image data acquired from the three-dimensional field of the subject; and the route designating means designates the route through a display screen image of the multi-screen reconstructed image constructed by the multi-screen reconstructed image constructing means.
8. The endoscope device according toclaim 7 , wherein: the three-dimensional image is a virtual VBS image that represents the intracorporeal lumen of the subject and that is constructed based on CT image data which is the image data acquired from the three-dimensional field of the subject; and the multi-screen reconstructed image is an MRT image that represents the intracorporeal lumen of the subject and that is constructed based on the CT image data.
9. The endoscope device according toclaim 8 , wherein the image processing unit constructs reduced images of three-dimensional images using the MRT image, which is constructed by the multi-screen reconstructed image constructing means, according to the result of calculation performed by the route designating means.
10. The endoscope device according toclaim 8 , wherein the three-dimensional image constructing means constructs successive VBS images, which represent regions along the route determined by the route designating unit, in units of a frame in synchronization with the endoscopic image.
11. The endoscope device according toclaim 8 , further comprising: a fetching unit that fetches the CT image data; a first storage unit in which the CT image data fetched by the fetching unit is stored; and a second storage unit in which image data representing the VBS images is stored.
12. The endoscope device according toclaim 3 , further comprising degree-of-insertion calculating means for calculating a degree of insertion to which the endoscope is inserted, wherein: the image display control means displays the endoscopic image, which is contained in the navigation image, according to the result of calculation of a degree of insertion performed by the degree-of-insertion calculating means.
13. The endoscope device according toclaim 3 , further comprising operational input means for use in giving an operational directive to the route designating means and image display control means.
14. The endoscope device according toclaim 3 , wherein the image display control means rotates the three-dimensional image contained in the navigation image displayed on a monitor, and displays it.
15. An endoscope device in accordance with the present invention, comprising:
three-dimensional image constructing means for constructing a three-dimensional image of an intracorporeal lumen of a subject on the basis of image data acquired from a three-dimensional field of the subject;
an endoscope that images the intracorporeal lumen of the subject;
first navigation image constructing means for constructing a navigation image that includes an endoscopic image, which is produced by the endoscope and represents the intracorporeal lumen of the subject, and the three-dimensional image;
route designating means for designating and calculating a route that starts with a point at which the endoscope is inserted and ends with a point at which the endoscope reaches a target region of the subject, and that is used to present the course of the endoscope to be steered in order to insert the endoscope;
second navigation image constructing means for constructing reduced images of three-dimensional images, which represent all branch points on the determined route at which the lumen branches out, according to the result of calculation performed by the route designating means, and for constructing the navigation image that has the reduced images added thereto; and
image display control means for displaying the navigation image, which is constructed by the second navigation image constructing means, on a monitor.
16. A navigation method implemented in an endoscope device in accordance with the present invention, comprising:
a three-dimensional image constructing procedure for constructing a three-dimensional image of an intracorporeal lumen of a subject on the basis of image data acquired from a three-dimensional field of the subject;
a first navigation image constructing procedure for constructing a navigation image that includes an endoscopic image, which represents the intracorporeal lumen of the subject and is produced by an endoscope that picks up images of an intracorporeal lumen of a subject, and the three-dimensional image;
a route designating procedure for designating and calculating a route that starts with a point at which the endoscope is inserted and ends with a point at which the endoscope reaches a target region of the subject, and that is used to present the course of the endoscope to be steered in order to insert the endoscope;
a second navigation image constructing procedure for constructing reduced images of three-dimensional images, which represent all branch points on the determined route at which the lumen branches out, according to the result of calculation performed according to the route designating procedure, and for constructing the navigation image that has the reduced images added thereto;
an image display control procedure for displaying the navigation image, which is constructed according to the second navigation image constructing procedure, on a monitor.
17. An endoscopic image display method, comprising:
an endoscopic image data reception step of receiving image data that represents a lumen of a subject and that is picked up by an endoscope;
a first subject image data construction step of constructing first subject image data, which represents a lumen, on the basis of three-dimensional data acquired from the subject;
a second subject image data construction step of constructing second subject image data, which represents a lumen, on the basis of the three-dimensional data representing the interior of the subject;
an endoscopic image display step of displaying the endoscopic image on the basis of the endoscopic image data;
a first subject image display step of displaying the first subject image on the basis of the first subject image data; and
a second subject image display step of displaying the second subject image on the basis of the second subject image data.
18. The endoscopic image display method according toclaim 17 , wherein the three-dimensional data acquired from the subject is data representing bronchi.
19. The endoscopic image display method according toclaim 18 , wherein an image representing a branch point of the bronchi is constructed at the first subject image data construction step.
20. The endoscopic image display method according toclaim 18 , wherein: images representing a plurality of different branch points of the bronchi are constructed at the second subject image data construction step; and the images representing the plurality of different branch points of the bronchi are displayed at the second subject image display step.
21. The endoscopic image display method according toclaim 20 , wherein at least part of the images representing the branch points of the bronchi displayed at the second subject image display step are displayed in the order of their being juxtaposed along the course of the endoscope to be steered in order to insert the endoscope.
22. An endoscopic image display program comprising:
an endoscopic image data reception step of receiving image data that represents a lumen of a subject and that is picked up by an endoscope;
a first subject image data construction step of constructing first subject image data, which represents a lumen, on the basis of three-dimensional data acquired from the subject;
a second subject image data construction step of constructing second subject image data, which represents a lumen, on the basis of three-dimensional data representing the interior of the subject;
an endoscopic image display step of displaying an endoscopic image on the basis of the endoscopic image data;
a first subject image display step of displaying the first subject image on the basis of the first subject image data; and
a second subject image display step of displaying a second subject image on the basis of the second subject image data.
23. The endoscopic image display program according toclaim 22 , wherein the three-dimensional data acquired from the subject is data representing bronchi.
24. The endoscopic image display program according toclaim 23 , wherein an image representing a branch point of the bronchi is constructed at the first subject image data construction step.
25. The endoscopic image display program according toclaim 23 , wherein: images representing a plurality of different branch points of the bronchi are constructed at the second subject image data construction step; and the images representing the plurality of different branch points of the bronchi are displayed at the second subject image display step.
26. The endoscopic image display program according toclaim 25 , wherein at least part of the images representing the branch points of the bronchi displayed at the second subject image display step are displayed in the order of their being juxtaposed along the course of the endoscope to be steered in order to insert the endoscope.
27. An endoscopic image display method comprising:
an endoscopic image data reception step of receiving image data that represents a lumen of a subject and that is picked up by an endoscope;
a first subject image data construction step of constructing first subject image data, which represents a lumen, on the basis of three-dimensional data acquired from the subject;
a second subject image data construction step of constructing second subject image data, which represents a lumen, on the basis of the three-dimensional data representing the interior of the subject; and
a display step of displaying an endoscopic image that is based on the endoscopic image data received at the endoscopic image data reception step, a first subject image that is constructed at the first subject image data construction step, and a second subject image that is constructed at the second subject image data construction step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US11/702,337US20070142705A1 (en) | 2002-07-31 | 2007-02-05 | Endoscope device |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2002-223618 | 2002-07-31 | ||
| JP2002223618 | 2002-07-31 | ||
| PCT/JP2003/009733WO2004010857A1 (en) | 2002-07-31 | 2003-07-31 | Endoscope |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US11/702,337DivisionUS20070142705A1 (en) | 2002-07-31 | 2007-02-05 | Endoscope device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20050020878A1true US20050020878A1 (en) | 2005-01-27 |
| US7641609B2 US7641609B2 (en) | 2010-01-05 |
Family
ID=31184975
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US10/498,155Expired - LifetimeUS7641609B2 (en) | 2002-07-31 | 2003-07-31 | Endoscope device and navigation method for endoscope device |
| US11/702,337AbandonedUS20070142705A1 (en) | 2002-07-31 | 2007-02-05 | Endoscope device |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US11/702,337AbandonedUS20070142705A1 (en) | 2002-07-31 | 2007-02-05 | Endoscope device |
Country Status (7)
| Country | Link |
|---|---|
| US (2) | US7641609B2 (en) |
| EP (1) | EP1466552B1 (en) |
| JP (1) | JP4009639B2 (en) |
| CN (1) | CN100364479C (en) |
| AT (1) | ATE349944T1 (en) |
| DE (1) | DE60310877T8 (en) |
| WO (1) | WO2004010857A1 (en) |
Cited By (46)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050182295A1 (en)* | 2003-12-12 | 2005-08-18 | University Of Washington | Catheterscope 3D guidance and interface system |
| US20060100482A1 (en)* | 2004-11-09 | 2006-05-11 | Hale Eric L | Method for dealing with singularities in gravity referenced endoscopic imaging |
| US20060178558A1 (en)* | 2005-02-07 | 2006-08-10 | Mitsuo Obata | Endoscope system |
| US20060189842A1 (en)* | 2005-02-14 | 2006-08-24 | Hoeg Hans D | Method for using variable direction of view endoscopy in conjunction with image guided surgical systems |
| US20060281971A1 (en)* | 2005-06-14 | 2006-12-14 | Siemens Corporate Research Inc | Method and apparatus for minimally invasive surgery using endoscopes |
| US20070086633A1 (en)* | 2005-09-23 | 2007-04-19 | Jan Boese | Method for supporting an interventional medical operation |
| US20070142707A1 (en)* | 2005-12-15 | 2007-06-21 | Microvision, Inc. | Method and apparatus for calibrating an endoscope system |
| US20070299309A1 (en)* | 2005-02-28 | 2007-12-27 | University Of Washington | Monitoring disposition of tethered capsule endoscope in esophagus |
| US20080058629A1 (en)* | 2006-08-21 | 2008-03-06 | University Of Washington | Optical fiber scope with both non-resonant illumination and resonant collection/imaging for multiple modes of operation |
| US20080086051A1 (en)* | 2006-09-20 | 2008-04-10 | Ethicon Endo-Surgery, Inc. | System, storage medium for a computer program, and method for displaying medical images |
| US20080132834A1 (en)* | 2006-12-04 | 2008-06-05 | University Of Washington | Flexible endoscope tip bending mechanism using optical fibers as tension members |
| US20080207997A1 (en)* | 2007-01-31 | 2008-08-28 | The Penn State Research Foundation | Method and apparatus for continuous guidance of endoscopy |
| US20080221434A1 (en)* | 2007-03-09 | 2008-09-11 | Voegele James W | Displaying an internal image of a body lumen of a patient |
| US20080221388A1 (en)* | 2007-03-09 | 2008-09-11 | University Of Washington | Side viewing optical fiber endoscope |
| US20080234566A1 (en)* | 2007-03-21 | 2008-09-25 | Ethicon Endo-Surgery, Inc. | Recognizing a real world fiducial in a patient image data |
| US20080232656A1 (en)* | 2007-03-22 | 2008-09-25 | Ethicon Endo-Surgery, Inc. | Recognizing a real world fiducial in image data of a patient |
| US20080234544A1 (en)* | 2007-03-20 | 2008-09-25 | Ethicon Endo-Sugery, Inc. | Displaying images interior and exterior to a body lumen of a patient |
| US20080243030A1 (en)* | 2007-04-02 | 2008-10-02 | University Of Washington | Multifunction cannula tools |
| US20080303898A1 (en)* | 2007-06-06 | 2008-12-11 | Olympus Medical Systems Corp. | Endoscopic image processing apparatus |
| US20080319307A1 (en)* | 2007-06-19 | 2008-12-25 | Ethicon Endo-Surgery, Inc. | Method for medical imaging using fluorescent nanoparticles |
| US20090024191A1 (en)* | 2006-03-03 | 2009-01-22 | University Of Washington | Multi-cladding optical fiber scanner |
| US20090054761A1 (en)* | 2007-08-22 | 2009-02-26 | Ethicon Endo-Surgery, Inc. | Medical system, method, and storage medium concerning a natural orifice transluminal medical procedure |
| US20090137893A1 (en)* | 2007-11-27 | 2009-05-28 | University Of Washington | Adding imaging capability to distal tips of medical tools, catheters, and conduits |
| US20090235396A1 (en)* | 2000-06-19 | 2009-09-17 | University Of Washington | Integrated optical scanning image acquisition and display |
| US20090323076A1 (en)* | 2007-05-03 | 2009-12-31 | University Of Washington | High resolution optical coherence tomography based imaging for intraluminal and interstitial use implemented with a reduced form factor |
| US20100204545A1 (en)* | 2007-11-29 | 2010-08-12 | Olympus Medical Systems Corp. | Endoscope system |
| US20100249506A1 (en)* | 2009-03-26 | 2010-09-30 | Intuitive Surgical, Inc. | Method and system for assisting an operator in endoscopic navigation |
| US20110184238A1 (en)* | 2010-01-28 | 2011-07-28 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
| US20110221756A1 (en)* | 2008-11-13 | 2011-09-15 | Hitachi Medical Corporation | Image processing device and image processing method |
| US8102416B2 (en) | 2010-02-22 | 2012-01-24 | Olympus Medical Systems Corp. | Medical apparatus |
| WO2012068194A3 (en)* | 2010-11-18 | 2012-08-02 | C2Cure Inc. | Endoscope guidance based on image matching |
| US20120203065A1 (en)* | 2011-02-04 | 2012-08-09 | The Penn State Research Foundation | Global and semi-global registration for image-based bronchoscopy guidance |
| US8537203B2 (en) | 2005-11-23 | 2013-09-17 | University Of Washington | Scanning beam with variable sequential framing using interrupted scanning resonance |
| US20140088357A1 (en)* | 2012-03-06 | 2014-03-27 | Olympus Medical Systems Corp. | Endoscope system |
| US8840566B2 (en) | 2007-04-02 | 2014-09-23 | University Of Washington | Catheter with imaging capability acts as guidewire for cannula tools |
| US20150150537A1 (en)* | 2012-08-08 | 2015-06-04 | Kabushiki Kaisha Toshiba | Medical image diagnosis apparatus, image processing apparatus, and image processing method |
| US20160317233A1 (en)* | 2014-04-23 | 2016-11-03 | Veran Medical Technologies, Inc. | Apparatuses and methods for registering a real-time image feed from an imaging device to a steerable catheter |
| US20180090176A1 (en)* | 2016-09-28 | 2018-03-29 | Fujifilm Corporation | Medical image storage and reproduction apparatus, method, and program |
| US10694929B2 (en) | 2014-12-15 | 2020-06-30 | Olympus Corporation | Medical equipment system and operation method of medical equipment system |
| US10765308B2 (en)* | 2009-06-29 | 2020-09-08 | Koninklijke Philips N.V. | Method and apparatus for tracking in a medical procedure |
| US10856770B2 (en) | 2009-03-26 | 2020-12-08 | Intuitive Surgical Operations, Inc. | Method and system for providing visual guidance to an operator for steering a tip of an endoscopic device towards one or more landmarks in a patient |
| US20210077211A1 (en)* | 2009-10-12 | 2021-03-18 | Corindus, Inc. | System and method for navigating a guide wire |
| US20210093406A1 (en)* | 2009-10-12 | 2021-04-01 | Corindus, Inc. | System and method for navigating a guide wire |
| US11188285B2 (en)* | 2014-07-02 | 2021-11-30 | Covidien Lp | Intelligent display |
| EP3919019A1 (en)* | 2020-06-03 | 2021-12-08 | Covidien LP | Surgical tool navigation using sensor fusion |
| US11555997B2 (en) | 2015-04-27 | 2023-01-17 | Endochoice, Inc. | Endoscope with integrated measurement of distance to objects of interest |
Families Citing this family (89)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7998062B2 (en) | 2004-03-29 | 2011-08-16 | Superdimension, Ltd. | Endoscope structures and techniques for navigating to a target in branched structure |
| EP2316328B1 (en) | 2003-09-15 | 2012-05-09 | Super Dimension Ltd. | Wrap-around holding device for use with bronchoscopes |
| EP2113189B1 (en) | 2003-09-15 | 2013-09-04 | Covidien LP | System of accessories for use with bronchoscopes |
| JP4573517B2 (en)* | 2003-10-29 | 2010-11-04 | オリンパス株式会社 | Insertion support system |
| EP2189107A3 (en) | 2003-10-31 | 2010-09-15 | Olympus Corporation | Insertion support system |
| JP3847744B2 (en) | 2003-11-04 | 2006-11-22 | オリンパス株式会社 | Insertion support system |
| US8764725B2 (en)* | 2004-02-09 | 2014-07-01 | Covidien Lp | Directional anchoring mechanism, method and applications thereof |
| US20050182319A1 (en) | 2004-02-17 | 2005-08-18 | Glossop Neil D. | Method and apparatus for registration, verification, and referencing of internal organs |
| KR100615881B1 (en)* | 2004-06-21 | 2006-08-25 | 한국과학기술연구원 | Capsule Endoscope Control System |
| US7517314B2 (en) | 2004-10-14 | 2009-04-14 | Karl Storz Development Corp. | Endoscopic imaging with indication of gravity direction |
| EP1827243B1 (en) | 2004-11-05 | 2010-01-20 | THE GOVERNMENT OF THE UNITED STATES OF AMERICA, as represented by THE SECRETARY, DEPARTMENT OF HEALTH AND HUMAN SERVICES | Access system |
| US7805269B2 (en) | 2004-11-12 | 2010-09-28 | Philips Electronics Ltd | Device and method for ensuring the accuracy of a tracking device in a volume |
| US7751868B2 (en) | 2004-11-12 | 2010-07-06 | Philips Electronics Ltd | Integrated skin-mounted multifunction device for use in image-guided surgery |
| US8611983B2 (en) | 2005-01-18 | 2013-12-17 | Philips Electronics Ltd | Method and apparatus for guiding an instrument to a target in the lung |
| WO2006076789A1 (en)* | 2005-01-24 | 2006-07-27 | Claron Technology Inc. | A bronchoscopy navigation system and method |
| US7221522B2 (en) | 2005-01-28 | 2007-05-22 | Karl Storz Development Corp. | Optical system for variable direction of view instrument |
| JP4776954B2 (en)* | 2005-03-17 | 2011-09-21 | オリンパス株式会社 | Endoscope insertion support device |
| CA2612603C (en) | 2005-06-21 | 2015-05-19 | Traxtal Inc. | Device and method for a trackable ultrasound |
| US8632461B2 (en) | 2005-06-21 | 2014-01-21 | Koninklijke Philips N.V. | System, method and apparatus for navigated therapy and diagnosis |
| DK1912671T3 (en) | 2005-07-18 | 2017-11-27 | Seattle Genetics Inc | BETA-glucuronide-LINKER-drug conjugates |
| US20070066881A1 (en) | 2005-09-13 | 2007-03-22 | Edwards Jerome R | Apparatus and method for image guided accuracy verification |
| EP1924198B1 (en) | 2005-09-13 | 2019-04-03 | Veran Medical Technologies, Inc. | Apparatus for image guided accuracy verification |
| DE602006015913D1 (en)* | 2006-04-10 | 2010-09-16 | Storz Karl Dev Corp | Endoscopic imaging with indication of the direction of gravity |
| WO2007129493A1 (en) | 2006-05-02 | 2007-11-15 | National University Corporation Nagoya University | Medical image observation support device |
| US8905920B2 (en)* | 2007-09-27 | 2014-12-09 | Covidien Lp | Bronchoscope adapter and method |
| EP2241922B1 (en)* | 2007-10-30 | 2012-02-22 | Olympus Corporation | Endoscope apparatus |
| EP2633810B1 (en) | 2008-02-12 | 2017-08-30 | Covidien LP | Controlled perspective guidance method |
| JP5291955B2 (en)* | 2008-03-10 | 2013-09-18 | 富士フイルム株式会社 | Endoscopy system |
| WO2009122273A2 (en)* | 2008-04-03 | 2009-10-08 | Superdimension, Ltd. | Magnetic interference detection system and method |
| EP2297673B1 (en) | 2008-06-03 | 2020-04-22 | Covidien LP | Feature-based registration method |
| US8218847B2 (en) | 2008-06-06 | 2012-07-10 | Superdimension, Ltd. | Hybrid registration method |
| US8932207B2 (en) | 2008-07-10 | 2015-01-13 | Covidien Lp | Integrated multi-functional endoscopic tool |
| US20100022824A1 (en) | 2008-07-22 | 2010-01-28 | Cybulski James S | Tissue modification devices and methods of using the same |
| US20110009694A1 (en)* | 2009-07-10 | 2011-01-13 | Schultz Eric E | Hand-held minimally dimensioned diagnostic device having integrated distal end visualization |
| US20100121139A1 (en) | 2008-11-12 | 2010-05-13 | Ouyang Xiaolong | Minimally Invasive Imaging Systems |
| DE102009010263B4 (en)* | 2009-02-24 | 2011-01-20 | Reiner Kunz | Method for navigating an endoscopic instrument during technical endoscopy and associated device |
| EP2413777B1 (en)* | 2009-04-03 | 2014-12-31 | Koninklijke Philips N.V. | Associating a sensor position with an image position |
| US8611984B2 (en)* | 2009-04-08 | 2013-12-17 | Covidien Lp | Locatable catheter |
| US9658760B2 (en)* | 2009-05-07 | 2017-05-23 | Creative Technology Ltd. | Methods for searching digital files on a user interface |
| JP2012525898A (en)* | 2009-05-08 | 2012-10-25 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Real-time scope tracking and branch labeling without electromagnetic tracking and preoperative roadmap scanning |
| JP5322794B2 (en)* | 2009-06-16 | 2013-10-23 | 株式会社東芝 | Endoscopy support system |
| US8696547B2 (en)* | 2009-09-17 | 2014-04-15 | Broncus Medical, Inc. | System and method for determining airway diameter using endoscope |
| JP5457841B2 (en)* | 2010-01-07 | 2014-04-02 | 株式会社東芝 | Medical image processing apparatus and medical image processing program |
| JP5362636B2 (en)* | 2010-03-31 | 2013-12-11 | 富士フイルム株式会社 | Medical air supply system |
| US10582834B2 (en) | 2010-06-15 | 2020-03-10 | Covidien Lp | Locatable expandable working channel and method |
| US8591401B2 (en)* | 2010-08-18 | 2013-11-26 | Olympus Corporation | Endoscope apparatus displaying information indicating gravity direction on screen |
| EP3659490B1 (en) | 2010-08-20 | 2025-10-01 | Veran Medical Technologies, Inc. | Apparatus and method for four dimensional soft tissue navigation |
| US9560953B2 (en) | 2010-09-20 | 2017-02-07 | Endochoice, Inc. | Operational interface in a multi-viewing element endoscope |
| JP5160699B2 (en)* | 2011-01-24 | 2013-03-13 | オリンパスメディカルシステムズ株式会社 | Medical equipment |
| US9020229B2 (en) | 2011-05-13 | 2015-04-28 | Broncus Medical, Inc. | Surgical assistance planning method using lung motion analysis |
| CN103298393B (en)* | 2011-05-17 | 2015-09-16 | 奥林巴斯医疗株式会社 | Labelling display control method in armarium, medical image and medical processor |
| JP2014209930A (en)* | 2011-08-31 | 2014-11-13 | テルモ株式会社 | Navigation system for respiration area |
| JP5977497B2 (en)* | 2011-09-22 | 2016-08-24 | オリンパス株式会社 | Endoscope apparatus, operation method and program |
| EP2816966B1 (en) | 2012-02-22 | 2023-10-25 | Veran Medical Technologies, Inc. | Steerable surgical catheter comprising a biopsy device at the distal end portion thereof |
| US20130250081A1 (en)* | 2012-03-21 | 2013-09-26 | Covidien Lp | System and method for determining camera angles by using virtual planes derived from actual images |
| CN109452930B (en) | 2012-05-14 | 2021-10-29 | 直观外科手术操作公司 | Registration system and method for medical devices using reduced search space |
| US10039473B2 (en) | 2012-05-14 | 2018-08-07 | Intuitive Surgical Operations, Inc. | Systems and methods for navigation based on ordered sensor records |
| JP6233913B2 (en)* | 2013-01-07 | 2017-11-22 | 晃 竹林 | Endoscope navigation device |
| US10499794B2 (en) | 2013-05-09 | 2019-12-10 | Endochoice, Inc. | Operational interface in a multi-viewing element endoscope |
| EP3777646B1 (en)* | 2013-05-09 | 2023-09-27 | EndoChoice, Inc. | Operational interface in a multi-viewing elements endoscope |
| US9875544B2 (en) | 2013-08-09 | 2018-01-23 | Broncus Medical Inc. | Registration of fluoroscopic images of the chest and corresponding 3D image data based on the ribs and spine |
| US11547446B2 (en) | 2014-01-13 | 2023-01-10 | Trice Medical, Inc. | Fully integrated, disposable tissue visualization device |
| US9370295B2 (en) | 2014-01-13 | 2016-06-21 | Trice Medical, Inc. | Fully integrated, disposable tissue visualization device |
| US10342579B2 (en) | 2014-01-13 | 2019-07-09 | Trice Medical, Inc. | Fully integrated, disposable tissue visualization device |
| US20150305650A1 (en) | 2014-04-23 | 2015-10-29 | Mark Hunter | Apparatuses and methods for endobronchial navigation to and confirmation of the location of a target tissue and percutaneous interception of the target tissue |
| US10952593B2 (en) | 2014-06-10 | 2021-03-23 | Covidien Lp | Bronchoscope adapter |
| US10426555B2 (en) | 2015-06-03 | 2019-10-01 | Covidien Lp | Medical instrument with sensor for use in a system and method for electromagnetic navigation |
| WO2017027749A1 (en) | 2015-08-11 | 2017-02-16 | Trice Medical, Inc. | Fully integrated, disposable tissue visualization device |
| JP6594133B2 (en)* | 2015-09-16 | 2019-10-23 | 富士フイルム株式会社 | Endoscope position specifying device, operation method of endoscope position specifying device, and endoscope position specifying program |
| JP6886968B2 (en)* | 2015-10-09 | 2021-06-16 | コヴィディエン リミテッド パートナーシップ | How to use an angled endoscope to visualize body cavities using a robotic surgical system |
| US10478254B2 (en) | 2016-05-16 | 2019-11-19 | Covidien Lp | System and method to access lung tissue |
| US10751126B2 (en) | 2016-10-28 | 2020-08-25 | Covidien Lp | System and method for generating a map for electromagnetic navigation |
| US10418705B2 (en) | 2016-10-28 | 2019-09-17 | Covidien Lp | Electromagnetic navigation antenna assembly and electromagnetic navigation system including the same |
| US10722311B2 (en) | 2016-10-28 | 2020-07-28 | Covidien Lp | System and method for identifying a location and/or an orientation of an electromagnetic sensor based on a map |
| US10446931B2 (en) | 2016-10-28 | 2019-10-15 | Covidien Lp | Electromagnetic navigation antenna assembly and electromagnetic navigation system including the same |
| US10615500B2 (en) | 2016-10-28 | 2020-04-07 | Covidien Lp | System and method for designing electromagnetic navigation antenna assemblies |
| US10792106B2 (en) | 2016-10-28 | 2020-10-06 | Covidien Lp | System for calibrating an electromagnetic navigation system |
| US10517505B2 (en) | 2016-10-28 | 2019-12-31 | Covidien Lp | Systems, methods, and computer-readable media for optimizing an electromagnetic navigation system |
| US10638952B2 (en) | 2016-10-28 | 2020-05-05 | Covidien Lp | Methods, systems, and computer-readable media for calibrating an electromagnetic navigation system |
| WO2018179986A1 (en)* | 2017-03-30 | 2018-10-04 | 富士フイルム株式会社 | Endoscope system, processor device, and method for operating endoscope system |
| WO2019078237A1 (en)* | 2017-10-18 | 2019-04-25 | 富士フイルム株式会社 | Medical image processing device, endoscope system, diagnosis supporting device, and medical business supporting system |
| US11219489B2 (en) | 2017-10-31 | 2022-01-11 | Covidien Lp | Devices and systems for providing sensors in parallel with medical tools |
| EP3773235B1 (en) | 2018-03-29 | 2023-07-19 | Trice Medical, Inc. | Fully integrated endoscope with biopsy capabilities |
| CN110742690A (en)* | 2019-09-12 | 2020-02-04 | 东南大学苏州医疗器械研究院 | Method for configuring endoscope and terminal equipment |
| CN111415564B (en)* | 2020-03-02 | 2022-03-18 | 武汉大学 | Navigation method and system for pancreatic endoscopic ultrasonography based on artificial intelligence |
| DE102021103411A1 (en) | 2021-02-12 | 2022-08-18 | Aesculap Ag | Surgical assistance system and display method |
| JP2023167710A (en) | 2022-05-12 | 2023-11-24 | 富士フイルム株式会社 | Information processing device, bronchus endoscope device, information processing method, and program |
| CN115797688A (en)* | 2022-11-28 | 2023-03-14 | 浙江优亿医疗器械股份有限公司 | Focus identification method and system based on bronchoscope |
| CN117221177B (en)* | 2023-11-08 | 2024-01-09 | 湖南省华芯医疗器械有限公司 | Image transmission delay monitoring method and system |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5417210A (en)* | 1992-05-27 | 1995-05-23 | International Business Machines Corporation | System and method for augmentation of endoscopic surgery |
| US5526812A (en)* | 1993-06-21 | 1996-06-18 | General Electric Company | Display system for enhancing visualization of body structures during medical procedures |
| US5765561A (en)* | 1994-10-07 | 1998-06-16 | Medical Media Systems | Video-based surgical targeting system |
| US5873822A (en)* | 1994-09-15 | 1999-02-23 | Visualization Technology, Inc. | Automatic registration system for use with position tracking and imaging system for use in medical applications |
| US6346940B1 (en)* | 1997-02-27 | 2002-02-12 | Kabushiki Kaisha Toshiba | Virtualized endoscope system |
| US6602185B1 (en)* | 1999-02-18 | 2003-08-05 | Olympus Optical Co., Ltd. | Remote surgery support system |
| US20030184598A1 (en)* | 1997-12-22 | 2003-10-02 | Ricoh Company, Ltd. | Television-based visualization and navigation interface |
| US6702736B2 (en)* | 1995-07-24 | 2004-03-09 | David T. Chen | Anatomical visualization system |
| US6801643B2 (en)* | 1995-06-01 | 2004-10-05 | Medical Media Systems | Anatomical visualization system |
| US20040205504A1 (en)* | 2001-12-05 | 2004-10-14 | International Business Machines Corporation | System and method for navigating graphocal images |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000135215A (en)* | 1998-10-30 | 2000-05-16 | Ge Yokogawa Medical Systems Ltd | Conduit guiding method and device thereof and radiation tomographic equipment |
| JP3850217B2 (en) | 2000-12-27 | 2006-11-29 | オリンパス株式会社 | Endoscope position detector for bronchi |
- 2003
- 2003-07-31JPJP2004524322Apatent/JP4009639B2/ennot_activeExpired - Lifetime
- 2003-07-31DEDE60310877Tpatent/DE60310877T8/enactiveActive
- 2003-07-31WOPCT/JP2003/009733patent/WO2004010857A1/enactiveIP Right Grant
- 2003-07-31EPEP03771449Apatent/EP1466552B1/ennot_activeExpired - Lifetime
- 2003-07-31CNCNB038020335Apatent/CN100364479C/ennot_activeExpired - Lifetime
- 2003-07-31USUS10/498,155patent/US7641609B2/ennot_activeExpired - Lifetime
- 2003-07-31ATAT03771449Tpatent/ATE349944T1/ennot_activeIP Right Cessation
- 2007
- 2007-02-05USUS11/702,337patent/US20070142705A1/ennot_activeAbandoned
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5417210A (en)* | 1992-05-27 | 1995-05-23 | International Business Machines Corporation | System and method for augmentation of endoscopic surgery |
| US5526812A (en)* | 1993-06-21 | 1996-06-18 | General Electric Company | Display system for enhancing visualization of body structures during medical procedures |
| US5873822A (en)* | 1994-09-15 | 1999-02-23 | Visualization Technology, Inc. | Automatic registration system for use with position tracking and imaging system for use in medical applications |
| US5765561A (en)* | 1994-10-07 | 1998-06-16 | Medical Media Systems | Video-based surgical targeting system |
| US6801643B2 (en)* | 1995-06-01 | 2004-10-05 | Medical Media Systems | Anatomical visualization system |
| US6702736B2 (en)* | 1995-07-24 | 2004-03-09 | David T. Chen | Anatomical visualization system |
| US6346940B1 (en)* | 1997-02-27 | 2002-02-12 | Kabushiki Kaisha Toshiba | Virtualized endoscope system |
| US20030184598A1 (en)* | 1997-12-22 | 2003-10-02 | Ricoh Company, Ltd. | Television-based visualization and navigation interface |
| US6602185B1 (en)* | 1999-02-18 | 2003-08-05 | Olympus Optical Co., Ltd. | Remote surgery support system |
| US20040205504A1 (en)* | 2001-12-05 | 2004-10-14 | International Business Machines Corporation | System and method for navigating graphocal images |
Cited By (89)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090235396A1 (en)* | 2000-06-19 | 2009-09-17 | University Of Washington | Integrated optical scanning image acquisition and display |
| US8396535B2 (en) | 2000-06-19 | 2013-03-12 | University Of Washington | Integrated optical scanning image acquisition and display |
| US20050182295A1 (en)* | 2003-12-12 | 2005-08-18 | University Of Washington | Catheterscope 3D guidance and interface system |
| US7901348B2 (en)* | 2003-12-12 | 2011-03-08 | University Of Washington | Catheterscope 3D guidance and interface system |
| US9554729B2 (en) | 2003-12-12 | 2017-01-31 | University Of Washington | Catheterscope 3D guidance and interface system |
| US9226687B2 (en)* | 2003-12-12 | 2016-01-05 | University Of Washington | Catheterscope 3D guidance and interface system |
| US8382662B2 (en) | 2003-12-12 | 2013-02-26 | University Of Washington | Catheterscope 3D guidance and interface system |
| US20130158346A1 (en)* | 2003-12-12 | 2013-06-20 | University Of Washington | Catheterscope 3D Guidance and Interface System |
| US7811224B2 (en)* | 2004-11-09 | 2010-10-12 | Karl Storz Development Corp. | Method for dealing with singularities in gravity referenced endoscopic imaging |
| US20110026787A1 (en)* | 2004-11-09 | 2011-02-03 | Eric Lawrence Hale | Method for dealing with singularities in gravity referenced endoscopic imaging |
| US20060100482A1 (en)* | 2004-11-09 | 2006-05-11 | Hale Eric L | Method for dealing with singularities in gravity referenced endoscopic imaging |
| US7662090B2 (en)* | 2005-02-07 | 2010-02-16 | Olympus Corporation | Endoscope system |
| US20060178558A1 (en)* | 2005-02-07 | 2006-08-10 | Mitsuo Obata | Endoscope system |
| US7967742B2 (en)* | 2005-02-14 | 2011-06-28 | Karl Storz Imaging, Inc. | Method for using variable direction of view endoscopy in conjunction with image guided surgical systems |
| US8414476B2 (en) | 2005-02-14 | 2013-04-09 | Karl Storz Imaging, Inc. | Method for using variable direction of view endoscopy in conjunction with image guided surgical systems |
| US20110230710A1 (en)* | 2005-02-14 | 2011-09-22 | Hans David Hoeg | Method For Using Variable Direction Of View Endoscopy In Conjunction With Image Guided Surgical Systems |
| US20060189842A1 (en)* | 2005-02-14 | 2006-08-24 | Hoeg Hans D | Method for using variable direction of view endoscopy in conjunction with image guided surgical systems |
| US9161684B2 (en) | 2005-02-28 | 2015-10-20 | University Of Washington | Monitoring disposition of tethered capsule endoscope in esophagus |
| US20070299309A1 (en)* | 2005-02-28 | 2007-12-27 | University Of Washington | Monitoring disposition of tethered capsule endoscope in esophagus |
| US9872613B2 (en) | 2005-02-28 | 2018-01-23 | University Of Washington | Monitoring disposition of tethered capsule endoscope in esophagus |
| US9289267B2 (en)* | 2005-06-14 | 2016-03-22 | Siemens Medical Solutions Usa, Inc. | Method and apparatus for minimally invasive surgery using endoscopes |
| US20060281971A1 (en)* | 2005-06-14 | 2006-12-14 | Siemens Corporate Research Inc | Method and apparatus for minimally invasive surgery using endoscopes |
| US20070086633A1 (en)* | 2005-09-23 | 2007-04-19 | Jan Boese | Method for supporting an interventional medical operation |
| US7860282B2 (en)* | 2005-09-23 | 2010-12-28 | Siemens Aktiengesellschaft | Method for supporting an interventional medical operation |
| US8537203B2 (en) | 2005-11-23 | 2013-09-17 | University Of Washington | Scanning beam with variable sequential framing using interrupted scanning resonance |
| US7547277B2 (en) | 2005-12-15 | 2009-06-16 | Microvision, Inc. | Method and apparatus for calibrating an endoscope system |
| US20070142707A1 (en)* | 2005-12-15 | 2007-06-21 | Microvision, Inc. | Method and apparatus for calibrating an endoscope system |
| WO2007070831A3 (en)* | 2005-12-15 | 2007-12-13 | Microvision Inc | Method and apparatus for calibrating an endoscope system |
| US9561078B2 (en) | 2006-03-03 | 2017-02-07 | University Of Washington | Multi-cladding optical fiber scanner |
| US20090024191A1 (en)* | 2006-03-03 | 2009-01-22 | University Of Washington | Multi-cladding optical fiber scanner |
| US20080058629A1 (en)* | 2006-08-21 | 2008-03-06 | University Of Washington | Optical fiber scope with both non-resonant illumination and resonant collection/imaging for multiple modes of operation |
| US20080086051A1 (en)* | 2006-09-20 | 2008-04-10 | Ethicon Endo-Surgery, Inc. | System, storage medium for a computer program, and method for displaying medical images |
| US20080132834A1 (en)* | 2006-12-04 | 2008-06-05 | University Of Washington | Flexible endoscope tip bending mechanism using optical fibers as tension members |
| US20080207997A1 (en)* | 2007-01-31 | 2008-08-28 | The Penn State Research Foundation | Method and apparatus for continuous guidance of endoscopy |
| US8672836B2 (en)* | 2007-01-31 | 2014-03-18 | The Penn State Research Foundation | Method and apparatus for continuous guidance of endoscopy |
| US20080221434A1 (en)* | 2007-03-09 | 2008-09-11 | Voegele James W | Displaying an internal image of a body lumen of a patient |
| US20080221388A1 (en)* | 2007-03-09 | 2008-09-11 | University Of Washington | Side viewing optical fiber endoscope |
| US20080234544A1 (en)* | 2007-03-20 | 2008-09-25 | Ethicon Endo-Sugery, Inc. | Displaying images interior and exterior to a body lumen of a patient |
| US8457718B2 (en) | 2007-03-21 | 2013-06-04 | Ethicon Endo-Surgery, Inc. | Recognizing a real world fiducial in a patient image data |
| US20080234566A1 (en)* | 2007-03-21 | 2008-09-25 | Ethicon Endo-Surgery, Inc. | Recognizing a real world fiducial in a patient image data |
| US20080232656A1 (en)* | 2007-03-22 | 2008-09-25 | Ethicon Endo-Surgery, Inc. | Recognizing a real world fiducial in image data of a patient |
| US8081810B2 (en) | 2007-03-22 | 2011-12-20 | Ethicon Endo-Surgery, Inc. | Recognizing a real world fiducial in image data of a patient |
| US8840566B2 (en) | 2007-04-02 | 2014-09-23 | University Of Washington | Catheter with imaging capability acts as guidewire for cannula tools |
| US20080243030A1 (en)* | 2007-04-02 | 2008-10-02 | University Of Washington | Multifunction cannula tools |
| US7952718B2 (en) | 2007-05-03 | 2011-05-31 | University Of Washington | High resolution optical coherence tomography based imaging for intraluminal and interstitial use implemented with a reduced form factor |
| US20090323076A1 (en)* | 2007-05-03 | 2009-12-31 | University Of Washington | High resolution optical coherence tomography based imaging for intraluminal and interstitial use implemented with a reduced form factor |
| US20080303898A1 (en)* | 2007-06-06 | 2008-12-11 | Olympus Medical Systems Corp. | Endoscopic image processing apparatus |
| US20080319307A1 (en)* | 2007-06-19 | 2008-12-25 | Ethicon Endo-Surgery, Inc. | Method for medical imaging using fluorescent nanoparticles |
| US8155728B2 (en) | 2007-08-22 | 2012-04-10 | Ethicon Endo-Surgery, Inc. | Medical system, method, and storage medium concerning a natural orifice transluminal medical procedure |
| US20090054761A1 (en)* | 2007-08-22 | 2009-02-26 | Ethicon Endo-Surgery, Inc. | Medical system, method, and storage medium concerning a natural orifice transluminal medical procedure |
| US20090137893A1 (en)* | 2007-11-27 | 2009-05-28 | University Of Washington | Adding imaging capability to distal tips of medical tools, catheters, and conduits |
| US8147402B2 (en)* | 2007-11-29 | 2012-04-03 | Olympus Medical Systems Corp. | Endoscope system |
| US20100204545A1 (en)* | 2007-11-29 | 2010-08-12 | Olympus Medical Systems Corp. | Endoscope system |
| US20110221756A1 (en)* | 2008-11-13 | 2011-09-15 | Hitachi Medical Corporation | Image processing device and image processing method |
| US8693759B2 (en)* | 2008-11-13 | 2014-04-08 | Hitachi Medical Corporation | Image processing device and image processing method |
| US11744445B2 (en) | 2009-03-26 | 2023-09-05 | Intuitive Surgical Operations, Inc. | Method and system for assisting an operator in endoscopic navigation |
| US10524641B2 (en) | 2009-03-26 | 2020-01-07 | Intuitive Surgical Operations, Inc. | Method and system for assisting an operator in endoscopic navigation |
| US10004387B2 (en)* | 2009-03-26 | 2018-06-26 | Intuitive Surgical Operations, Inc. | Method and system for assisting an operator in endoscopic navigation |
| US10856770B2 (en) | 2009-03-26 | 2020-12-08 | Intuitive Surgical Operations, Inc. | Method and system for providing visual guidance to an operator for steering a tip of an endoscopic device towards one or more landmarks in a patient |
| US20100249506A1 (en)* | 2009-03-26 | 2010-09-30 | Intuitive Surgical, Inc. | Method and system for assisting an operator in endoscopic navigation |
| US10765308B2 (en)* | 2009-06-29 | 2020-09-08 | Koninklijke Philips N.V. | Method and apparatus for tracking in a medical procedure |
| US20210077211A1 (en)* | 2009-10-12 | 2021-03-18 | Corindus, Inc. | System and method for navigating a guide wire |
| US20210093406A1 (en)* | 2009-10-12 | 2021-04-01 | Corindus, Inc. | System and method for navigating a guide wire |
| US11696808B2 (en)* | 2009-10-12 | 2023-07-11 | Corindus, Inc. | System and method for navigating a guide wire |
| US11918314B2 (en)* | 2009-10-12 | 2024-03-05 | Corindus, Inc. | System and method for navigating a guide wire |
| WO2011094518A3 (en)* | 2010-01-28 | 2011-11-10 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
| US10667679B2 (en)* | 2010-01-28 | 2020-06-02 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
| US20110184238A1 (en)* | 2010-01-28 | 2011-07-28 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
| US20180220883A1 (en)* | 2010-01-28 | 2018-08-09 | The Penn State Research Foundation | Image-based global registration system and method applicable to bronchoscopy guidance |
| US8102416B2 (en) | 2010-02-22 | 2012-01-24 | Olympus Medical Systems Corp. | Medical apparatus |
| WO2012068194A3 (en)* | 2010-11-18 | 2012-08-02 | C2Cure Inc. | Endoscope guidance based on image matching |
| US20120203065A1 (en)* | 2011-02-04 | 2012-08-09 | The Penn State Research Foundation | Global and semi-global registration for image-based bronchoscopy guidance |
| WO2012106320A1 (en)* | 2011-02-04 | 2012-08-09 | The Penn State Research Foundation | Global and semi-global registration for image-based bronchoscopy guidance |
| US9757021B2 (en)* | 2011-02-04 | 2017-09-12 | The Penn State Research Foundation | Global and semi-global registration for image-based bronchoscopy guidance |
| US8894566B2 (en)* | 2012-03-06 | 2014-11-25 | Olympus Medical Systems Corp. | Endoscope system |
| US20140088357A1 (en)* | 2012-03-06 | 2014-03-27 | Olympus Medical Systems Corp. | Endoscope system |
| US10123780B2 (en)* | 2012-08-08 | 2018-11-13 | Toshiba Medical Systems Corporation | Medical image diagnosis apparatus, image processing apparatus, and image processing method |
| US20150150537A1 (en)* | 2012-08-08 | 2015-06-04 | Kabushiki Kaisha Toshiba | Medical image diagnosis apparatus, image processing apparatus, and image processing method |
| US11553968B2 (en) | 2014-04-23 | 2023-01-17 | Veran Medical Technologies, Inc. | Apparatuses and methods for registering a real-time image feed from an imaging device to a steerable catheter |
| US20160317233A1 (en)* | 2014-04-23 | 2016-11-03 | Veran Medical Technologies, Inc. | Apparatuses and methods for registering a real-time image feed from an imaging device to a steerable catheter |
| US10624701B2 (en)* | 2014-04-23 | 2020-04-21 | Veran Medical Technologies, Inc. | Apparatuses and methods for registering a real-time image feed from an imaging device to a steerable catheter |
| US11188285B2 (en)* | 2014-07-02 | 2021-11-30 | Covidien Lp | Intelligent display |
| US11793389B2 (en) | 2014-07-02 | 2023-10-24 | Covidien Lp | Intelligent display |
| US10694929B2 (en) | 2014-12-15 | 2020-06-30 | Olympus Corporation | Medical equipment system and operation method of medical equipment system |
| US11555997B2 (en) | 2015-04-27 | 2023-01-17 | Endochoice, Inc. | Endoscope with integrated measurement of distance to objects of interest |
| US11056149B2 (en)* | 2016-09-28 | 2021-07-06 | Fujifilm Corporation | Medical image storage and reproduction apparatus, method, and program |
| US20180090176A1 (en)* | 2016-09-28 | 2018-03-29 | Fujifilm Corporation | Medical image storage and reproduction apparatus, method, and program |
| EP3919019A1 (en)* | 2020-06-03 | 2021-12-08 | Covidien LP | Surgical tool navigation using sensor fusion |
| US12064191B2 (en) | 2020-06-03 | 2024-08-20 | Covidien Lp | Surgical tool navigation using sensor fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1612708A (en) | 2005-05-04 |
| DE60310877D1 (en) | 2007-02-15 |
| EP1466552B1 (en) | 2007-01-03 |
| EP1466552A4 (en) | 2005-10-05 |
| ATE349944T1 (en) | 2007-01-15 |
| JP4009639B2 (en) | 2007-11-21 |
| DE60310877T8 (en) | 2007-09-13 |
| US20070142705A1 (en) | 2007-06-21 |
| DE60310877T2 (en) | 2007-05-24 |
| JPWO2004010857A1 (en) | 2005-11-24 |
| CN100364479C (en) | 2008-01-30 |
| WO2004010857A1 (en) | 2004-02-05 |
| US7641609B2 (en) | 2010-01-05 |
| EP1466552A1 (en) | 2004-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7641609B2 (en) | Endoscope device and navigation method for endoscope device | |
| EP1685787B1 (en) | Insertion support system | |
| JP3847744B2 (en) | Insertion support system | |
| US8509877B2 (en) | Endoscope insertion support system and endoscope insertion support method | |
| JP3930423B2 (en) | Endoscope device | |
| JP4343723B2 (en) | Insertion support system | |
| EP1681011B1 (en) | Insertion support system | |
| WO2004023986A1 (en) | Medical treatment system, endoscope system, endoscope insert operation program, and endoscope device | |
| JP4022114B2 (en) | Endoscope device | |
| JP4323288B2 (en) | Insertion support system | |
| JP4245880B2 (en) | Endoscope device | |
| JP4022192B2 (en) | Insertion support system | |
| JP4160487B2 (en) | Insertion support system | |
| JP2005304937A (en) | Insertion support system | |
| JP4546064B2 (en) | Insertion simulation device | |
| JP4573517B2 (en) | Insertion support system | |
| JP4575143B2 (en) | Insertion support system | |
| JP4354353B2 (en) | Insertion support system | |
| JP4190454B2 (en) | Insertion support device | |
| JP2006255193A (en) | Endoscope insertion support device | |
| JP2005013358A (en) | Insertion support system | |
| CN118285721A (en) | Endoscope system and data storage method for endoscope system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment | Owner name:OLYMPUS CORPORATION, JAPAN Free format text:ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:OHNISHI, JUNICHI;AKIMOTO, SHUNYA;KOBAYSHI, EIICHI;REEL/FRAME:016236/0374 Effective date:20040224 | |
| AS | Assignment | Owner name:OLYMPUS CORPORATION, JAPAN Free format text:RE-RECORD TO FIX A TYPOGRAPHICAL ERROR IN CONVEYING PARTY (3), ON A DOCUMENT PREVIOUSLY RECORDED AT REEL 016236, FRAME 0374. (ASSIGNMENT OF ASSIGNOR'S INTEREST);ASSIGNORS:OHNISHI, JUNICHI;AKIMOTO, SHUNYA;KOBAYASHI, EIICHI;REEL/FRAME:016605/0486 Effective date:20040224 | |
| AS | Assignment | Owner name:OLYMPUS CORPORATION, JAPAN Free format text:ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ASANO, FUMIHIRO;MORIYA, HIROSHI;REEL/FRAME:019536/0312;SIGNING DATES FROM 20070614 TO 20070620 | |
| FEPP | Fee payment procedure | Free format text:PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY | |
| STCF | Information on status: patent grant | Free format text:PATENTED CASE | |
| FPAY | Fee payment | Year of fee payment:4 | |
| AS | Assignment | Owner name:OLYMPUS CORPORATION, JAPAN Free format text:CHANGE OF ADDRESS;ASSIGNOR:OLYMPUS CORPORATION;REEL/FRAME:039344/0502 Effective date:20160401 | |
| FPAY | Fee payment | Year of fee payment:8 | |
| MAFP | Maintenance fee payment | Free format text:PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment:12 |