Three-dimensional reconstruction method, device and system based on single cameraTechnical Field
The invention relates to the technical field of three-dimensional reconstruction, in particular to a three-dimensional reconstruction method, a three-dimensional reconstruction device and a three-dimensional reconstruction system based on a single camera.
Background
Three-dimensional reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of a three-dimensional object, is the basis for processing, operating and analyzing the properties of the three-dimensional object in a computer environment, and is also a key technology for establishing virtual reality expressing an objective world in a computer.
In computer vision, three-dimensional reconstruction refers to the process of reconstructing three-dimensional information from single-view or multi-view images. Since the information of a single video is incomplete, the three-dimensional reconstruction needs to utilize empirical knowledge. The method is that the camera is calibrated, namely the relation between the image coordinate system of the camera and the world coordinate system is calculated, and then three-dimensional information is reconstructed by utilizing the information in a plurality of two-dimensional images.
In the existing three-dimensional reconstruction process, a monocular system mainly comprises a camera and a projector, wherein the projector mainly projects a phase shift diagram, and the camera mainly acquires the phase shift diagram modulated by the surface of an object. The system structure is that the projector is obliquely arranged and the camera is vertically arranged, so that after the projector projects periodic stripes, the stripes shot by the camera are not of equal width, and the depth of field of the camera in three-dimensional reconstruction is small and the requirement of high precision cannot be met in the height direction.
Disclosure of Invention
In view of the above, the present invention provides a three-dimensional reconstruction method, apparatus and system based on a single camera, which can solve the problems in the prior art that the depth of field of the camera is small and the requirement of high precision cannot be met in the height direction.
In a first aspect, an embodiment of the present invention provides a three-dimensional reconstruction method based on a single camera, including:
acquiring system parameters of a monocular system; the monocular system comprises a projector and a camera;
correcting the fringes projected by the projector according to the system parameters so that the projector projects a corrected first phase shift map;
receiving a second phase shift diagram which is acquired by a camera and obtained by modulating the first phase shift diagram by an object to be scanned;
performing phase demodulation, phase unwrapping and height mapping processing on the second phase shift map to obtain a depth map;
and performing three-dimensional reconstruction on the object to be scanned according to the depth map.
With reference to the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where acquiring a system parameter of a monocular system includes:
acquiring a plurality of groups of point pair coordinates (x, y) obtained by projecting an image in a fixed mode by a projector;
substituting the multiple groups of point pair coordinates (x, y) into a first preset formula, and calculating to obtain system parameters; the system parameters include: the included angle theta between the projection optical axis and the normal line of the reference plane, the distance L between the projection optical center and the reference plane and the equivalent focal length d of the projection system.
With reference to the first aspect, an embodiment of the present invention provides a second possible implementation manner of the first aspect, where correcting the fringes projected by the projector according to the system parameters includes:
generating an equiperiodic sinogram by a projector;
determining the relation between the coordinate x and the coordinate y according to the system parameters;
and based on the relation between the coordinate x and the coordinate y, the sinogram of the peer-to-peer period is transformed to obtain a corrected phase shift graph serving as a first phase shift graph.
With reference to the first aspect, an embodiment of the present invention provides a third possible implementation manner of the first aspect, where performing phase demodulation, phase unwrapping, and height mapping on the second phase shift map to obtain a depth map includes:
performing phase demodulation processing on the second phase shift diagram according to a second preset formula to obtain a main phase;
performing phase unwrapping processing on the main phase to obtain an unwrapped phase;
and calculating to obtain the height value of the surface of the object according to the expansion phase and a third preset formula, and taking the set of the height values as a depth map.
With reference to the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where performing phase demodulation processing on the second phase shift diagram according to a second preset formula to obtain a main phase includes:
calculating the modulation phase of the grating stripe corresponding to the point pair coordinate (x, y) according to the second phase shift diagram and a second preset formula
Will modulate the phaseAs the primary phase.
With reference to the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, where performing phase unwrapping processing on a main phase to obtain an unwrapped phase includes:
constructing a Poisson equation based on the main phase;
and solving the Poisson equation by utilizing discrete Fourier transform to obtain the unwrapped phase.
With reference to the first aspect, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, where three-dimensional reconstruction is performed on an object to be scanned according to a depth map, and the method includes:
obtaining three-dimensional point cloud of the object based on the depth map; the three-dimensional coordinates of the three-dimensional point cloud correspond to a plurality of groups of point pair coordinates and corresponding height values thereof;
and performing three-dimensional reconstruction of the object to be scanned according to the three-dimensional point cloud.
In a second aspect, an embodiment of the present invention further provides a three-dimensional reconstruction apparatus based on a single camera, including:
the parameter acquisition module is used for acquiring system parameters of the monocular system; the monocular system comprises a projector and a camera;
the fringe correction module is used for correcting the fringes projected by the projector according to the system parameters so as to enable the projector to project the corrected first phase shift diagram;
the phase shift diagram receiving module is used for receiving a second phase shift diagram which is acquired by a camera and obtained by modulating the first phase shift diagram by an object to be scanned;
the depth map acquisition module is used for carrying out phase demodulation, phase unwrapping and height mapping processing on the second phase shift map to obtain a depth map;
and the three-dimensional reconstruction module is used for performing three-dimensional reconstruction on the object to be scanned according to the depth map.
In a third aspect, an embodiment of the present invention further provides a three-dimensional reconstruction system based on a single camera, including: a server and a monocular system;
the monocular system includes: a projector and a camera;
the server is provided with the single-camera-based three-dimensional reconstruction device in the second aspect;
the server is in communication connection with the monocular system.
In a fourth aspect, the present invention also provides a computer readable medium having non-volatile program code executable by a processor, where the program code causes the processor to execute the method according to the first aspect.
The embodiment of the invention has the following beneficial effects:
the three-dimensional reconstruction method based on the single camera provided by the embodiment of the invention comprises the following steps of firstly, obtaining system parameters of a monocular system, wherein the monocular system comprises a projector and a camera; correcting the fringes projected by the projector according to the system parameters so that the projector projects a corrected first phase shift map; receiving a second phase shift diagram which is acquired by a camera and obtained by modulating the first phase shift diagram by an object to be scanned; performing phase demodulation, phase unwrapping and height mapping processing on the second phase shift map to obtain a depth map; and performing three-dimensional reconstruction on the object to be scanned according to the depth map. According to the invention, the periodic stripes are corrected by obtaining the system parameters of the calibrated monocular system, so that the high-precision height value is obtained through height mapping subsequently, and the problems that the depth of field of the camera is small and the requirement on high precision cannot be met in the height direction in the prior art are solved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a three-dimensional reconstruction method based on a single camera according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a monocular system in a three-dimensional reconstruction method based on a single camera according to an embodiment of the present invention;
fig. 3 is a projection coordinate diagram in a three-dimensional reconstruction method based on a single camera according to a first embodiment of the present invention;
fig. 4 is a flowchart of another single-camera-based three-dimensional reconstruction method according to an embodiment of the present invention;
fig. 5 is a schematic diagram of coordinates in a checkerboard in a three-dimensional reconstruction method based on a single camera according to an embodiment of the present invention;
fig. 6 is a flowchart of another single-camera-based three-dimensional reconstruction method according to an embodiment of the present invention;
fig. 7 is a first phase shift diagram of a projector projection in a three-dimensional reconstruction method based on a single camera according to an embodiment of the present invention.
Fig. 8 is a second phase shift diagram acquired by a camera in a three-dimensional reconstruction method based on a single camera according to an embodiment of the present invention;
fig. 9 is a flowchart of another single-camera-based three-dimensional reconstruction method according to an embodiment of the present invention;
fig. 10 is a schematic phase unwrapping diagram in a three-dimensional reconstruction method based on a single camera according to an embodiment of the present invention;
fig. 11 is a schematic diagram of high mapping in a three-dimensional reconstruction method based on a single camera according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of a three-dimensional reconstruction apparatus based on a single camera according to a second embodiment of the present invention;
fig. 13 is a schematic structural diagram of a flowchart system of a three-dimensional reconstruction method based on a single camera according to a third embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the existing three-dimensional reconstruction process, a monocular system mainly comprises a camera and a projector, wherein the projector mainly projects a phase shift diagram, and the camera mainly acquires the phase shift diagram modulated by the surface of an object. The system structure is that the projector is obliquely arranged and the camera is vertically arranged, so that after the projector projects periodic stripes, the stripes shot by the camera are not of equal width, and the depth of field of the camera in three-dimensional reconstruction is small and the requirement of high precision cannot be met in the height direction.
Based on this, the embodiment of the invention provides a three-dimensional reconstruction method, a three-dimensional reconstruction device and a three-dimensional reconstruction system based on a single camera, which correct periodic stripes by obtaining system parameters of a calibrated monocular system, so that high-precision height values can be obtained through height mapping subsequently, and the problems that the depth of field of the camera is small and the requirement on high precision cannot be met in the height direction in the prior art are solved.
In order to facilitate understanding of the embodiment, a three-dimensional reconstruction method based on a single camera disclosed in the embodiment of the present invention is first described in detail.
The first embodiment is as follows:
the embodiment of the invention provides a three-dimensional reconstruction method based on a single camera, which is applied to a server and shown in figure 1, and comprises the following steps:
s11: and acquiring system parameters of the monocular system.
Wherein the monocular system comprises a projector and a camera, as shown in fig. 2. The projector mainly projects phase shift maps and the camera mainly collects phase shift maps modulated by the object surface. Because the projector is obliquely arranged and the camera is vertically arranged in the system, the stripes shot by the camera are not equal in width after the projector projects the stripes with the same period. To solve this problem, some system parameters need to be calibrated to correct for streaks. The embodiment obtains the system parameters of the monocular system through the calibration system. The system parameters to be calibrated are an included angle theta between a projection optical axis and a normal line of a reference plane, a distance L between the projection optical center and the reference plane, and an equivalent focal length d of the projection system, and a projection schematic diagram is shown in fig. 3.
The process for acquiring system parameters of the monocular system mainly includes the following steps, as shown in fig. 4:
s111: a plurality of sets of point pair coordinates (x, y) obtained by projecting an image of a fixed pattern by a projector are acquired.
(x, y) may be obtained by projecting a fixed pattern of patterns, say a checkerboard, to find a series of (x, y) point pair coordinates. As shown in fig. 5, the coordinates in the projected projection grid are x, and the coordinates of the corresponding reference plane captured by the camera are y.
S112: and substituting the multiple groups of point pair coordinates (x, y) into a first preset formula, and calculating to obtain monocular system parameters.
Wherein, the first preset formula is as follows:
the calibration process is to calculate three unknowns of the system, namely L, d and theta, according to the coordinates of multiple groups of (x, y) point pairs. The system parameters are obtained by substituting the coordinates (x, y) of the point pairs into the formula. The system parameters include: the included angle theta between the projection optical axis and the normal line of the reference plane, the distance L between the projection optical center and the reference plane and the equivalent focal length d of the projection system.
S12: and correcting the fringes projected by the projector according to the system parameters so that the projector projects the corrected first phase shift diagram.
The specific calibration process includes the following steps, as shown in fig. 6:
s121: an equi-periodic sinogram is generated by a projector.
S122: and determining the relation between the coordinate x and the coordinate y according to the system parameters.
S123: and based on the relation between the coordinate x and the coordinate y, the sinogram of the peer-to-peer period is transformed to obtain a corrected phase shift graph serving as a first phase shift graph.
When the method is concretely realized, firstly, an equal-period sinogram is generated through a projector, then the generated equal-period sinogram is transformed according to a relation between a coordinate x and a coordinate y determined by the system parameters, and the obtained graph is a corrected graph which needs to be projected by the projector, namely a first phase shift graph. Fig. 7 shows a four-step phase shift diagram.
S13: and receiving a second phase shift diagram which is acquired by the camera and obtained by modulating the first phase shift diagram by the object to be scanned.
After the projector projects the first phase shift map, the camera acquires a phase shift map modulated by the object surface. The object is an object to be scanned. The object modulation process is as follows: projecting the sine grating on the surface of an object to be scanned, wherein the distribution of the stripes of the sine grating is changed on the surface of the object, namely, phase modulation is carried out on the sine projection, the modulated phase comprises height information of the surface of the object, the phase relation between points on the surface of the object is independent, and if the phases of the points can be obtained, the height information of the surface of the object can be obtained. The first phase shift diagram shown in fig. 7 is modulated by the object to be scanned to obtain a second phase shift diagram, and as shown in fig. 8, the server receives the second phase shift diagram collected by the camera for subsequent processing.
S14: and carrying out phase demodulation, phase unwrapping and height mapping processing on the second phase shift map to obtain a depth map.
The above processing procedure specifically includes the following steps, as shown in fig. 9:
s141: and carrying out phase demodulation processing on the second phase shift diagram according to a second preset formula to obtain a main phase.
The four-step phase shift has four phase shift graphs, and phase demodulation can be carried out by the four phase shift graphs to obtain a phase graph. Specifically, the modulation phase of the grating stripe corresponding to the point pair coordinate (x, y) is calculated according to the second phase shift diagram and a second preset formulaWill modulate the phaseAs the primary phase.
The second preset formula, i.e. the formula of phase demodulation, is as follows:
wherein, IB(x, y) is the background intensity independent of the phase-shifted fringe projected, IR(x, y) is the reflected light intensity of the measured point (x, y) to the projection light,for the modulation phase value of the measured point (x, y) to the projection light, 0, pi/2, pi, 3 pi/2 in the formula refers to the initial phase value of the quartic phase shift. I is1(x,y),I2(x,y),I3(x,y),I4(x, y) refers to the object image after phase modulation as taken by the camera, i.e., the second phase shift map. Solving the equation system can obtain the modulation phase of the (x, y) point on the image to the grating fringe, as shown in the following formula:
s142: and performing phase unwrapping processing on the main phase to obtain an unwrapped phase.
And after the second phase shift diagram is subjected to phase demodulation processing to obtain a main phase, further performing unwrapping process, and firstly constructing a Poisson equation based on the main phase. And then solving the Poisson equation by utilizing discrete Fourier transform to obtain the unwrapped phase.
The range of the main phase is [ -pi, pi ], which is a wrapped phase, i.e., a phase with periodicity. To obtain the true height of the object surface, the periodic phase must be transformed to a continuous phase, a process called phase unwrapping. As shown in figure 10 of the drawings,
and constructing a wrapping operator and a difference operator in the x direction and the y direction.
Wherein,and W {. is a wrapping operator for the difference operator in the x direction and the y direction. Rhoi,jThe intermediate quantities calculated from the wrapped phases are represented for use in the next solution to the unwrapped phases. And solving the Poisson equation constructed in the first step by utilizing discrete Fourier transform to obtain the phase after unwrapping.
The relationship between the unwrapped phase and the intermediate amount ρ (i, j) is as follows:
(φi+1,j-2φi,j+φi-1,j)+(φi,j+1-2φi,j+φi,j-1)＝ρi,j；
where φ represents the unwrapped phase that needs to be solved.
First calculating p in the last stepi,jDiscrete fourier transform of Pu,vThen, the intermediate variable Q is calculated according to the following formulau,v。
Finally calculate Qu,vInverse discrete Fourier transform of the order of the phase of the input signal to obtain an unwrapped phase phii,j。
S143: and calculating to obtain the height value of the surface of the object according to the expansion phase and a third preset formula, and taking the set of the height values as a depth map.
The height map is shown in fig. 11, which shows that, according to the geometric relationship,
where P is the period, which has been corrected before, i.e. P is a constant, the projected light incident angles β corresponding to different points in the field of view are different, and therefore the surface heights are also different, so as to obtain the height of the object surface, i.e. the depth map.
S15: and performing three-dimensional reconstruction on the object to be scanned according to the depth map.
Specifically, based on the depth map, obtaining a three-dimensional point cloud of the object; the three-dimensional coordinates of the three-dimensional point cloud correspond to a plurality of groups of point pair coordinates and corresponding height values thereof; and further performing three-dimensional reconstruction on the object to be scanned according to the three-dimensional point cloud.
The obtained depth map is specific to each pixel point, namely a z value corresponding to different point pair coordinates (x, y) is obtained, and the x, y and z are combined according to a corresponding sequence to obtain a three-dimensional point cloud, so that three-dimensional reconstruction is completed.
According to the three-dimensional reconstruction method based on the single camera, provided by the embodiment of the invention, the periodic stripes are corrected by obtaining the system parameters of the calibrated monocular system, so that the high-precision height value is obtained through height mapping subsequently, the problems that the depth of field of the camera is small and the requirement on high precision cannot be met in the height direction in the prior art are solved, and the calibration method is simple and easy compared with other methods, the calibration difficulty and complexity are reduced, and the efficiency is improved.
The three-dimensional reconstruction method based on the single camera provided by the embodiment of the invention has the following advantages:
in the calibration algorithm, system parameters aiming at a monocular system are deduced and calibrated based on basic parameters of a camera projector, and height mapping is carried out according to the calibrated system parameters. Compared with other methods, the calibration method is simple and easy to implement, the calibration difficulty and complexity are reduced, and the efficiency is improved. Specifically, the method comprises the following steps:
1. the calibration of the camera and the internal parameters of the projection adopts a Zhangyingyou calibration method, the detailed description is omitted, the calibration of the external parameters of the system only needs a checkerboard projected by a projector, and the operation and the calculation are simple;
2. the system parameters are calibrated and used for correcting periodic stripes, and even if the projector is obliquely arranged, the stripes projected on the reference plane are all in equal periods, so that the modeling difficulty is greatly simplified during height mapping calculation;
3. according to research, many calibration limiting conditions of other monocular systems are high, for example, some systems need to ensure the perpendicularity of the reference plane relative to the camera, some systems need to ensure that the optical center of the camera projection needs to be on a line parallel to the reference plane, and some systems have requirements on the projection and the included angle of the optical axis of the camera.
Example two:
an embodiment of the present invention further provides a three-dimensional reconstruction apparatus based on a single camera, as shown in fig. 12, the apparatus includes: a parameter acquisition module 21, a streak correction module 22, a phase shift map receiving module 23, a depth map acquisition module 24, and a three-dimensional reconstruction module 25.
The parameter acquiring module 21 is configured to acquire system parameters of a monocular system; the monocular system comprises a projector and a camera; a fringe correction module 22, configured to correct a fringe projected by the projector according to the system parameter, so that the projector projects the corrected first phase shift map; a phase shift diagram receiving module 23, configured to receive a second phase shift diagram acquired by the camera and obtained by modulating the first phase shift diagram with the object to be scanned; the depth map obtaining module 24 is configured to perform phase demodulation, phase unwrapping, and height mapping on the second phase shift map to obtain a depth map; and the three-dimensional reconstruction module 25 is configured to perform three-dimensional reconstruction on the object to be scanned according to the depth map.
In the three-dimensional reconstruction device based on the single camera provided by the embodiment of the invention, each module has the same technical characteristics as the three-dimensional reconstruction method based on the single camera, so that the functions can be realized. The specific working process of each module in the device refers to the above method embodiment, and is not described herein again.
Example three:
an embodiment of the present invention further provides a three-dimensional reconstruction system based on a single camera, as shown in fig. 13, the system includes: a server 31 and a monocular system 32.
Among them, monocular system 32 includes: a projector 321 and a camera 322; the server 31 is provided with a single-camera based three-dimensional reconstruction device 311 as described in the second aspect; server 31 is communicatively coupled to monocular system 32.
In the three-dimensional reconstruction system based on the single camera provided by the embodiment of the invention, each module has the same technical characteristics as the three-dimensional reconstruction device based on the single camera, so that the functions can be realized. The specific working process of each module in the system refers to the above method embodiment, and is not described herein again.
The computer program product of the single-camera-based three-dimensional reconstruction method provided in the embodiment of the present invention includes a computer-readable storage medium storing a non-volatile program code executable by a processor, where instructions included in the program code may be used to execute the method described in the foregoing method embodiment, and specific implementation may refer to the method embodiment, and will not be described herein again.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the apparatus and the electronic device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.