


技术领域technical field
本发明涉及一种三维重建方法,更进一步,涉及一种基于四步相移编码型面结构光的三维重建方法。The invention relates to a three-dimensional reconstruction method, and furthermore, to a three-dimensional reconstruction method based on four-step phase-shift coding profile structured light.
背景技术Background technique
随着人工智能技术与计算机科学的不断发展,如何实现对对象的智能感知与快速测量成为亟待解决的技术问题。为了解决这个问题,大量不同的三维测量技术在各个行业的得到广泛应用,如工业流水线产品尺寸检测、逆向工程、虚拟现实等。由于各类三维测量的实现原理不同,其相对之间具有不同的技术特点。总的来说,三维测量技术总体发展的目标为:高精度、高速度、大尺度范围。With the continuous development of artificial intelligence technology and computer science, how to realize intelligent perception and rapid measurement of objects has become a technical problem that needs to be solved urgently. In order to solve this problem, a large number of different 3D measurement technologies have been widely used in various industries, such as industrial assembly line product size inspection, reverse engineering, virtual reality, etc. Due to the different realization principles of various 3D measurements, they have different technical characteristics. In general, the overall development goals of 3D measurement technology are: high precision, high speed, and large scale range.
传统的三维测量技术可以分为两个大类:接触式三维测量与非接触式三维测量。接触式三维测量通过与被测物体的实际接触实现测量,是目前三维测量领域较为经典的技术形式。非接触式三维测量技术通过无线介质实现对物体的三维测量,如声、光、电磁等。其中三维光学测量技术相对于其他非接触式三维测量技术,具有精度高、易实现、速度快等特点,因此三维光学测量得到了大量的应用,同时发展出了不同的技术形式。Traditional 3D measurement techniques can be divided into two categories: contact 3D measurement and non-contact 3D measurement. Contact 3D measurement realizes measurement through actual contact with the object to be measured, and is a relatively classic technical form in the field of 3D measurement at present. Non-contact 3D measurement technology realizes 3D measurement of objects through wireless media, such as sound, light, electromagnetic, etc. Among them, 3D optical measurement technology has the characteristics of high precision, easy implementation and fast speed compared with other non-contact 3D measurement technologies. Therefore, 3D optical measurement has been widely used, and different technical forms have been developed.
其中,基于面结构光(Structed light)的三维测量是三维光学测量技术中的典型代表。通过一个或多个工业相机与投影仪的组合,结构光法可以实现高精度、快速的三维测量。此外,结构光系统还具有成本低、系统配置简单等特点,便于控制成本和大规模的应用。结构光三维测量的基本技术过程是:使用计算机预制一组具有特定编码的结构光图案,并经过投影仪将其投射至被测物体表面,通过相机的同步采集,得到变形的结构光图案。通过提取被采集图像中的相位信息,结合标定得到的相机-投影仪系统光学模型,可以反解出被测物体的三维信息。作为结构光法的代表,传统的相移测量轮廓术(Phase ShiftingProfilometry)投射多幅具有相等间距相位移的正弦光栅图案来实现对物体的三维测量。为了准确的提取相位信息和展开包裹相位,单次测量常常需要投射12幅光栅(3频4步)。虽然相移测量轮廓术在测量精度上较其他结构光法具有巨大优势,但大量的待投射图片限制了相移测量轮廓术在高速、实时测量环境中的应用。Among them, the three-dimensional measurement based on surface structured light (Structed light) is a typical representative in the three-dimensional optical measurement technology. Through the combination of one or more industrial cameras and projectors, the structured light method can achieve high-precision and fast three-dimensional measurement. In addition, the structured light system also has the characteristics of low cost and simple system configuration, which is convenient for cost control and large-scale application. The basic technical process of structured light 3D measurement is: use a computer to prefabricate a set of structured light patterns with specific codes, project them onto the surface of the object to be measured through a projector, and obtain a deformed structured light pattern through synchronous acquisition by cameras. By extracting the phase information in the collected images, combined with the optical model of the camera-projector system obtained by calibration, the three-dimensional information of the measured object can be inversely solved. As a representative of the structured light method, the traditional Phase Shifting Profilometry (Phase Shifting Profilometry) projects a plurality of sinusoidal grating patterns with equal spacing and phase shift to realize the three-dimensional measurement of the object. In order to accurately extract the phase information and unfold the wrapped phase, a single measurement often needs to project 12 gratings (3 frequencies and 4 steps). Although phase-shift measurement profilometry has great advantages over other structured light methods in measurement accuracy, a large number of images to be projected limit the application of phase-shift measurement profilometry in high-speed, real-time measurement environments.
发明内容SUMMARY OF THE INVENTION
本发明的目的在于提供一种基于四步相移编码型面结构光的三维重建方法,该方法的主要内容是生成四幅具有特殊编码方式的相移编码型面结构光,并通过本发明提出的相位解算算法对被采集图像进行相位信息的重新解算而得到绝对相位信息,将得到的绝对相位信息引入预先标定建立的系统三维坐标-相位对应模型,从而得到测物体表面的三维信息。该基于四步相移编码型面结构光的三维重建方法仅仅需要投射四张图片来完成对物体的一次三维测量,并最终得到与传统方法同样精度的三维测量结果。同时该方法可实现与被捕捉图像像素密度相当的测量密度,得到具有高分辨率的三维测量结果。The purpose of the present invention is to provide a three-dimensional reconstruction method based on the four-step phase-shift coding profile structured light, the main content of the method is to generate four phase shift coding profile structured lights with special coding methods, The phase calculation algorithm recalculates the phase information of the acquired image to obtain the absolute phase information, and the obtained absolute phase information is introduced into the pre-calibrated and established system three-dimensional coordinate-phase correspondence model, so as to obtain the three-dimensional information of the surface of the measuring object. The three-dimensional reconstruction method based on the four-step phase-shift coding surface structured light only needs to project four pictures to complete a three-dimensional measurement of the object, and finally obtain a three-dimensional measurement result with the same accuracy as the traditional method. At the same time, the method can achieve a measurement density equivalent to the pixel density of the captured image, and obtain a three-dimensional measurement result with high resolution.
为了达到上述目的,本发明提供的技术方案是:一种基于四步相移编码型面结构光的三维重建方法,该方法具体包括以下步骤:In order to achieve the above object, the technical solution provided by the present invention is: a three-dimensional reconstruction method based on a four-step phase-shift coding surface structured light, the method specifically includes the following steps:
步骤1:生成四步相移编码的结构光图案,其表达式如下:Step 1: Generate a four-step phase-shift encoded structured light pattern whose expression is as follows:
在上述4式中,为像素点(x,y)的光强,p代表投影仪,n=1,2,3,4为对应的编码步数。A为生成光栅的平均光强系数,B为生成光栅的光强幅值调制系数。φ(x,y)为点(x,y)的所编码的绝对相位信息,α(x,y)为点(x,y)的所编码的相移信息。其中α(x,y)按如下方式编码:In the above 4 equations, is the light intensity of the pixel point (x, y), p represents the projector, and n=1, 2, 3, and 4 are the corresponding coding steps. A is the average light intensity coefficient of the generated grating, and B is the light intensity amplitude modulation coefficient of the generated grating. φ(x,y) is the encoded absolute phase information of the point (x,y), and α(x,y) is the encoded phase shift information of the point (x,y). where α(x,y) is encoded as follows:
上式中N为生成光栅设定的条纹周期总数,k(x,y)为点(x,y)的级数,范围为[1,N]。In the above formula, N is the total number of fringe periods set by the generated grating, k(x, y) is the series of points (x, y), and the range is [1, N].
步骤2:将生成的四步面结构光图案输入投影仪,通过投影仪将四幅图案依次投射到被测物体上,同时相机同步采集该四幅因为物体表面调制而发生变形的结构光图案。四幅被采集图案的光强可以表示为:Step 2: Input the generated four-step structured light pattern into the projector, project the four patterns on the object to be measured in sequence through the projector, and simultaneously collect the four structured light patterns that are deformed due to the modulation of the object surface. The light intensity of the four collected patterns can be expressed as:
在上述四式中,为被采集图像像素点(x,y)的光强,c代表相机。A′为采集光栅的平均光强系数,B′为采集光栅的光强幅值调制系数。In the above four formulas, is the light intensity of the acquired image pixel (x, y), and c represents the camera. A' is the average light intensity coefficient of the collecting grating, and B' is the light intensity amplitude modulation coefficient of the collecting grating.
步骤3:基于步骤2中相机采集得到的四幅图像,按照下式分别提取被采集图像中包含的相位信息与相移信息:Step 3: Based on the four images collected by the camera in
其中为包裹相位。in for the wrapping phase.
步骤4:基于3中获得的包裹相位与相移α(x,y),求解得到被捕捉图像中包含的绝对相位。通过相移α(x,y)可求解得到条纹周期对应的条纹级数:Step 4: Based on the wrapped phase obtained in 3 With the phase shift α(x,y), the solution obtains the absolute phase contained in the captured image. The fringe series corresponding to the fringe period can be obtained by solving the phase shift α(x, y):
其中Round函数为四舍五入取整函数。然后依据式求得的条纹级数,可以解得绝对相位:The Round function is a rounding function. Then according to the fringe series obtained by the formula, the absolute phase can be solved:
步骤5:通过相位信息与预先标定的投影仪-相机系统数学模型计算物体表面点的三维坐标数据,即可完成待测物体三维重建。Step 5: The three-dimensional reconstruction of the object to be measured can be completed by calculating the three-dimensional coordinate data of the surface points of the object through the phase information and the pre-calibrated mathematical model of the projector-camera system.
作为对本发明的该四步相移编码型结构光三维重建方法的进一步优选的实施例,所述步骤1中点(x,y)的所编码的绝对相位信息φ(x,y)在竖直条纹光栅中表示为:φ(x,y)=2πx/h,在水平条纹光栅中表示为:φ(x,y)=2πy/h,其中h为一个周期条纹的节距。点(x,y)的所编码的级数信息k(x,y)在竖直条纹光栅中表示为:k(x,y)=int(x/h),在水平条纹光栅中表示为:k(x,y)=int(y/h),其中int为向上取整函数。As a further preferred embodiment of the four-step phase-shift encoding type structured light three-dimensional reconstruction method of the present invention, the encoded absolute phase information φ(x, y) at the point (x, y) in the
附图说明Description of drawings
图1是本发明的基于四步相移编码型面结构光的三维重建方法的测量系统示意图。FIG. 1 is a schematic diagram of the measurement system of the three-dimensional reconstruction method based on the four-step phase-shift coding profile structured light of the present invention.
图2是本发明的基于四步相移编码型面结构光的三维重建方法的流程示意图。FIG. 2 is a schematic flowchart of the three-dimensional reconstruction method based on the four-step phase-shift coding profile structured light of the present invention.
图3是本发明的基于四步相移编码型面结构光的三维重建方法生成的该四步相移编码型面结构光示意图。FIG. 3 is a schematic diagram of the four-step phase-shift coding profile structured light generated by the three-dimensional reconstruction method based on the four-step phase-shift coding profile structured light of the present invention.
具体实施方式Detailed ways
以下描述用于揭露本发明以使本领域技术人员能够实现本发明。以下描述中的优选实施例只作为举例,本领域技术人员可以想到其他显而易见的变型。在以下描述中界定的本发明的基本原理可以应用于其他实施方案、变形方案、改进方案、等同方案以及没有背离本发明的精神和范围的其他技术方案。The following description serves to disclose the invention to enable those skilled in the art to practice the invention. The preferred embodiments described below are given by way of example only, and other obvious modifications will occur to those skilled in the art. The basic principles of the invention defined in the following description may be applied to other embodiments, variations, improvements, equivalents, and other technical solutions without departing from the spirit and scope of the invention.
结合附图1至附图3,本发明公开一种基于四步相移编码型面结构光的三维重建方法,其通过一个投影仪-相机测量系统来实现。该测量系统包括一台计算机1、一台相机2、一台投影仪3以及被测物体4。该计算机1分别与相机2和投影仪3相连,能够控制该投影仪3向该被测量物体一次投射包含相位信息的四步相移编码的结构光条纹光栅图像。该工业相机2能够在计算机1的控制下同步采集经过该被测物体4表面调制的条纹光栅图像,并将其发送到该计算机1以进行相位相移解算和后续三维重建的操作。被测物体4位于该相机2和该投影仪3的前方,该布置位置应保证让被测特征被投影仪3相机1的公共投影区域所覆盖。With reference to Fig. 1 to Fig. 3, the present invention discloses a three-dimensional reconstruction method based on four-step phase-shift coding profile structured light, which is realized by a projector-camera measurement system. The measurement system includes a
结合附图所示,本发明公开一种基于四步相移编码型面结构光的三维重建方法。具体实现步骤如下:As shown in the accompanying drawings, the present invention discloses a three-dimensional reconstruction method based on a four-step phase-shift coding surface structured light. The specific implementation steps are as follows:
步骤1:如图1所示的结构光测量系统,主要由该计算机1、该相机2、该投影仪3与该被测物体4组成。该结构光测量系统的布置应尽量保证让投影仪3与相机1的公共视场足够大,且被测物体的被测特征应被该投影仪3所投射的光栅所覆盖。例如在附图1中,该投影仪3与该相机2分别为被测物体的右侧和左侧,二者的光轴相交,投影区域相互重叠,且该被测物体4正好位于二者的公共视场内。布置好硬件之后,标定该结构光测量系统,建立投影仪-相机的内外参数模型。Step 1: The structured light measurement system shown in FIG. 1 is mainly composed of the
步骤2:由该计算机1生成四步相移编码光栅。根据投影仪投影元件分辨率生成对应长度H与对应宽度W的四步相移光栅;根据测量需要选定整幅光栅图像的条纹数N;根据测量需要选择是竖直条纹光栅或水平条纹光栅,并计算得到需要生成的条纹节距h;根据测量需要确定平均光强系数A与光强幅值调制系数B。例如在附图3,生成H=768像素与W=1024像素的包含十个条纹周期的竖直条纹光栅,其平均光强系数A=0.5,光强幅值调制系数B=0.5。Step 2: The
步骤3:由该计算机1将步骤2中生成的四步相移编码结构光栅通过该投影仪3依次投影至物体表面,同时经由该相机2同步采集,采集得到变形之后的条纹光栅图像。对应的四幅被采集图像存储于该计算机1。Step 3: The
步骤4:基于步骤3中采集得到的四幅变形的结构光栅图案,分别依据如下公式计算每一个像素点的包裹相位与相移α(x,y):Step 4: Based on the four deformed structured grating patterns collected in
步骤5:基于步骤4中计算得到的相移值α(x,y),依据如下公式分别求解得到每一个像素点级数值:Step 5: Based on the phase shift value α(x, y) calculated in
步骤6:基于步骤4中计算得到的各像素点的包裹相位值和步骤5中计算得到的各像素点的级数值k(x,y),依据如下公式计算得到图像中各个像素点的绝对相位值φ(x,y):Step 6: Based on the wrapped phase value of each pixel calculated in
步骤7:基于步骤6中的计算得到的各点的绝对相位值与步骤1中标定得到的该结构光系统的投影仪-相机的内外参数模型,求解各个像素点对应物体表面的三维空间坐标值,完成对物体的三维重建与测量。Step 7: Based on the absolute phase value of each point obtained by calculation in Step 6 and the internal and external parameter model of the projector-camera of the structured light system obtained by calibration in
总的来说,本发明提供的一种基于四步相移编码型面结构光的三维重建方法的技术思路是通过生成四幅相位与相移同时编码的面结构光图像,从而实现对物体的三维形貌进行测量。本发明同时提供对应的解相以及解包裹算法,可直接由四张被采集的编码图像中恢复出对应像素点的绝对相位值,从而最终得到物体的三维测量结果。In general, the technical idea of a three-dimensional reconstruction method based on four-step phase-shift coding surface structured light provided by the present invention is to generate four surface structured light images encoded with phase and phase shift at the same time, so as to realize the three-dimensional reconstruction of the object. The shape is measured. The present invention simultaneously provides the corresponding unpacking and unwrapping algorithms, which can directly recover the absolute phase value of the corresponding pixel point from the four collected coded images, thereby finally obtaining the three-dimensional measurement result of the object.
上述实施例仅用于说明本发明,各部件的结构、尺寸、设置位置及形状都是可以有所变化的,在本发明技术方案的基础上,凡根据本发明原理对个别部件进行的改进和等同变换,均不应排除在本发明的保护范围之外。以上对本发明的一个实施例进行了详细说明,但所述内容仅为本发明的较佳实施例,不能被认为用于限定本发明的实施范围。凡依本发明申请范围所作的均等变化与改进等,均应仍归属于本发明的专利涵盖范围之内。The above embodiments are only used to illustrate the present invention, and the structure, size, setting position and shape of each component can be changed to some extent. On the basis of the technical solution of the present invention, any improvement and Equivalent transformation shall not be excluded from the protection scope of the present invention. An embodiment of the present invention has been described in detail above, but the content is only a preferred embodiment of the present invention, and cannot be considered to limit the scope of the present invention. All equivalent changes and improvements made according to the scope of the application of the present invention should still belong to the scope of the patent of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910481636.5ACN110174079B (en) | 2019-06-04 | 2019-06-04 | A three-dimensional reconstruction method based on four-step phase-shift coding surface structured light |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910481636.5ACN110174079B (en) | 2019-06-04 | 2019-06-04 | A three-dimensional reconstruction method based on four-step phase-shift coding surface structured light |
| Publication Number | Publication Date |
|---|---|
| CN110174079A CN110174079A (en) | 2019-08-27 |
| CN110174079Btrue CN110174079B (en) | 2020-11-06 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910481636.5AActiveCN110174079B (en) | 2019-06-04 | 2019-06-04 | A three-dimensional reconstruction method based on four-step phase-shift coding surface structured light |
| Country | Link |
|---|---|
| CN (1) | CN110174079B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110660093A (en)* | 2019-09-17 | 2020-01-07 | 上海工程技术大学 | Roller press roller surface reconstruction device and method based on structured light |
| CN111102939B (en)* | 2019-12-04 | 2021-11-02 | 天津大学 | Separation method of superimposed phase-shift grating in double-projection structured light vision 3D measurement |
| CN112361993A (en)* | 2020-12-08 | 2021-02-12 | 北京工业大学 | Three-dimensional reconstruction method and system based on three-step phase shift method and Flood Fill method |
| CN113012277B (en)* | 2021-02-03 | 2022-06-21 | 中国地质大学(武汉) | A multi-camera reconstruction method based on DLP surface structured light |
| CN113008164A (en)* | 2021-03-23 | 2021-06-22 | 南京理工大学 | Rapid high-precision three-dimensional surface shape reconstruction method |
| CN115507773B (en)* | 2021-06-07 | 2025-02-18 | 广州视源电子科技股份有限公司 | Three-dimensional measurement method and system |
| CN114463490B (en)* | 2021-12-30 | 2025-05-09 | 浙江大华技术股份有限公司 | Three-dimensional model construction method, device, terminal and computer-readable storage medium |
| WO2024113127A1 (en)* | 2022-11-29 | 2024-06-06 | 电子科技大学 | Three-dimensional reconstruction method based on structured light and binocular vision |
| CN116222443A (en)* | 2023-04-24 | 2023-06-06 | 湖南华曙高科技股份有限公司 | Single-frequency four-step phase-shift three-dimensional measurement method based on structured light |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100554869C (en)* | 2007-07-11 | 2009-10-28 | 华中科技大学 | A kind of two dimension three based on color structured light is separated phase measuring method frequently |
| CN101109616A (en)* | 2007-08-02 | 2008-01-23 | 华中科技大学 | A Three-Frequency Heterodyne Phase Shift Phase Decomposition Method |
| CN102155924B (en)* | 2010-12-17 | 2012-07-04 | 南京航空航天大学 | Four-step phase shifting method based on absolute phase recovery |
| CN102589479B (en)* | 2012-03-06 | 2014-11-26 | 天津大学 | Three-dimensional topography central perturbation composite grating projection measuring method and device |
| CN102881043A (en)* | 2012-09-18 | 2013-01-16 | 天津工业大学 | High-precision three-dimensional reconstruction method applied to scenes with large light and shade contrast |
| CN104103058B (en)* | 2013-04-01 | 2018-08-14 | 鸿富锦精密工业(深圳)有限公司 | Image matching system and method |
| CN106839977B (en)* | 2016-12-23 | 2019-05-07 | 西安科技大学 | Real-time measurement method of shield muck volume based on grating projection binocular imaging technology |
| CN106840036B (en)* | 2016-12-30 | 2019-10-01 | 浙江四点灵机器人股份有限公司 | A kind of diadactic structure light optimization method suitable for fast three-dimensional appearance measuring |
| CN107036556B (en)* | 2017-04-28 | 2019-12-13 | 南昌航空大学 | Three-dimensional measurement method of structured light based on segmented quantization phase encoding |
| CN108613637B (en)* | 2018-04-13 | 2020-04-07 | 深度创新科技(深圳)有限公司 | Structured light system dephasing method and system based on reference image |
| CN109523627B (en)* | 2018-10-24 | 2023-07-18 | 重庆大学 | A 3D Reconstruction Method Based on Taylor Exponential Expression Surface Structured Light |
| CN109186476B (en)* | 2018-10-26 | 2021-01-26 | 广东工业大学 | Color structured light three-dimensional measurement method, device, equipment and storage medium |
| CN109323669B (en)* | 2018-11-22 | 2025-03-14 | 上海交通大学医学院附属第九人民医院 | Laser stripe projection device |
| CN109443239A (en)* | 2018-12-03 | 2019-03-08 | 广州欧科信息技术股份有限公司 | Structural light measurement method, apparatus, equipment, storage medium and system |
| Publication number | Publication date |
|---|---|
| CN110174079A (en) | 2019-08-27 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110174079B (en) | A three-dimensional reconstruction method based on four-step phase-shift coding surface structured light | |
| CN109253708B (en) | A deep learning-based fringe projection temporal phase unwrapping method | |
| CN109523627B (en) | A 3D Reconstruction Method Based on Taylor Exponential Expression Surface Structured Light | |
| CN108955571A (en) | The method for three-dimensional measurement that double frequency heterodyne is combined with phase-shift coding | |
| CN102519390B (en) | Three coding period gray scale trapezoid phase shift structured light three dimensional information obtaining method | |
| CN108534714A (en) | Based on sinusoidal and binary system fringe projection quick three-dimensional measurement method | |
| CN110411374B (en) | A dynamic three-dimensional surface measurement method and system | |
| CN111174730B (en) | A Fast Phase Unwrapping Method Based on Phase Encoding | |
| CN102261896A (en) | Method and system for measuring three-dimensional shape of object based on phase measurement | |
| CN109945802A (en) | A kind of structural light three-dimensional measurement method | |
| CN110207620A (en) | A kind of three-dimensional rebuilding method determining digital fringe projection structure optical orders by different frequency | |
| CN113587852A (en) | Color fringe projection three-dimensional measurement method based on improved three-step phase shift | |
| CN105300317A (en) | Three-dimensional measurement method on the basis of projection of sinusoidal stripes and triangular wave stripes | |
| CN112945089A (en) | Structured light coding method based on stripe width modulation | |
| CN113345039B (en) | Three-dimensional reconstruction quantization structure optical phase image coding method | |
| CN112802084B (en) | Three-dimensional shape measurement method, system and storage medium based on deep learning | |
| CN112146596A (en) | An Optimal Quantized Phase Coding 3D Measurement Method | |
| CN111947600B (en) | Robust three-dimensional phase unfolding method based on phase level cost filtering | |
| CN104697468B (en) | Optical three-dimensional measuring method based on slanted phase strip encoding | |
| CN104501741A (en) | Orthogonal grating phase shift method for three-dimensional surface shape measurement | |
| CN112945086A (en) | Structured light coding method based on space sequence and light intensity threshold segmentation | |
| CN105698708B (en) | A kind of 3D vision method for reconstructing | |
| KR101001894B1 (en) | 3D shape measurement device and method using color projection moire technique | |
| CN110428459A (en) | A method of the Phase- un- wrapping based on numerical order coding | |
| CN102519396B (en) | Three-dimensional information acquisition method for sampling points of three gray level symmetrical linear coding periods |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |