CN113012269A - Three-dimensional image data rendering method and equipment based on GPU - Google Patents

Abstract

Translated fromChinese

本发明公开了一种基于GPU的三维图像数据渲染方法，其包括：获取三维图像数据并计算顶点数据，设置顶点数组对象和顶点缓冲对象；基于GLSL编辑顶点着色器、几何着色器和片元着色器并加载在GPU中；在GPU中由顶点着色器读入顶点数据并发送给几何着色器；在GPU中由几何着色器根据顶点数据进行图元信息计算，所述图元信息计算包括顶点坐标空间转换、法向量空间转换、光照计算；在GPU中由片元着色器根据图元信息计算结果计算图元的颜色值并绑定顶点数组对象，将处理后的图元信息写入显卡缓存区；根据显卡缓存区中的图元信息进行可视化表达。本发明提供的三维图像数据渲染方法能够提升三维图像重建过程中与用户交互性能以及提高图像可视化效果。

The invention discloses a GPU-based three-dimensional image data rendering method, which comprises: acquiring three-dimensional image data and calculating vertex data, setting vertex array objects and vertex buffer objects; editing vertex shaders, geometry shaders and fragment shaders based on GLSL In the GPU, the vertex shader reads the vertex data and sends it to the geometry shader; in the GPU, the geometry shader performs primitive information calculation according to the vertex data, and the primitive information calculation includes vertex coordinates Space conversion, normal vector space conversion, lighting calculation; in the GPU, the fragment shader calculates the color value of the primitive according to the calculation result of the primitive information and binds the vertex array object, and writes the processed primitive information into the graphics card buffer area ; Visually express according to the primitive information in the graphics card buffer area. The three-dimensional image data rendering method provided by the present invention can improve the interaction performance with the user in the three-dimensional image reconstruction process and the image visualization effect.

Description

Three-dimensional image data rendering method and equipment based on GPU

Technical Field

The invention relates to a three-dimensional visualization rendering technology in computer graphics, in particular to a three-dimensional image data rendering method and equipment based on a GPU.

Background

With the continuous development of computer hardware platforms, the scale of numerical calculation simulation is also continuously increased, the generated data volume also reaches TB and even PB magnitude, and the visualization processing of mass data is a hot and difficult problem of the current visualization research. Since the isosurface extraction method is a common method for capturing typical characteristics of mass data, in recent years, the isosurface extraction method of mass data is always the focus of research in the field of visualization at home and abroad. Among a plurality of isosurface extraction methods, the Marching Cubes algorithm becomes a recognized standard algorithm for extracting isosurface of mass data at present by virtue of the advantages of simple principle and easy realization, and is widely applied in various application fields.

The Marching Cube algorithm is a classical algorithm for surface rendering, which is a voxel-based three-dimensional reconstruction algorithm, and is also called an Isosurface Extraction algorithm (Isosurface Extraction). The Marching cube algorithm extracts the isosurface of each voxel, firstly sets the threshold of the surface to be extracted based on the gray value of the vertex of each cubic voxel, and divides the vertex of each voxel into two types which are larger than the threshold and smaller than the threshold according to the threshold, and finally extracts the surface. There are more or less than two cases for each of the 8 vertices of a cube, so there are 256 extracted surfaces in total, and the number can be reduced to 15 according to the symmetry of the cube.

Image data rendering is an important step in an image visualization process, and many of the existing methods are based on an OpenGL fixed rendering pipeline (such as an implementation in a VTK). With the development of three-dimensional rendering technology, the use of a Graphics Processing Unit (GPU) greatly accelerates the efficiency and effect of real-time rendering of three-dimensional Graphics. It has been demonstrated that GPUs can provide tens to hundreds of times the performance of CPUs in terms of floating point operations, parallel computations, and other parts of graphics rendering related computations. The existing fixed rendering pipeline technology based on OpenGL does not fully exert the performance of a GPU, when a threshold value changes, a complete calculation process needs to be carried out again, and the interactivity with a user is poor.

Disclosure of Invention

In view of the defects of the prior art, the invention provides a three-dimensional image data rendering method and equipment based on a GPU (graphics processing unit) so as to improve the interaction performance with a user in the three-dimensional image reconstruction process and improve the image visualization effect and performance.

In order to achieve the purpose, the invention adopts the following technical scheme:

a method of GPU-based three-dimensional image data rendering, comprising:

acquiring three-dimensional image data, calculating vertex data, and setting a vertex group object and a vertex buffer object;

editing a vertex shader, a geometry shader and a fragment shader based on the GLSL and loading the vertex shader, the geometry shader and the fragment shader into the GPU;

reading vertex data by a vertex shader in a GPU and sending the vertex data to a geometry shader;

performing primitive information calculation by a geometric shader according to vertex data in a GPU, wherein the primitive information calculation comprises vertex coordinate space conversion, normal vector space conversion and illumination calculation;

calculating the color value of a pixel according to the pixel information calculation result by a fragment shader in the GPU, binding a vertex group object, and writing the processed pixel information into a display card cache region;

and performing visual expression according to the graphic element information in the display card buffer area.

The method for acquiring the three-dimensional image data is to acquire the three-dimensional image data by using a Marching Cubes algorithm, and comprises the following steps:

reading the area information of the whole area corresponding to the object to be processed into a three-dimensional array;

extracting a voxel from the three-dimensional array to become a current voxel and acquiring vertex information of the voxel;

comparing the value of the vertex of the current voxel with the value of the given isosurface to obtain the state of the voxel;

finding out a voxel edge intersected with the isosurface according to the state index of the current voxel, and calculating the position coordinate of each intersection point by adopting a linear interpolation method;

and calculating a plane passing through each edge intersection point in the volume element, and taking the plane normal direction and each edge intersection point as a vertex to obtain an integral patch and acquire integral three-dimensional image data.

The GPU is also provided with a primitive assembling module, the vertex data read by the vertex shader is firstly sent to the primitive assembling module, the primitive assembling module assembles the vertex data into a primitive with a specified shape according to the vertex data and then sends the primitive to the geometry shader, and the geometry shader performs primitive information calculation according to the vertex data in the form of the primitive.

Wherein the primitive of the specified shape is a point, a line or a triangle.

And the geometry shader calculates the primitive information according to the vertex data and the instruction parameters input by the user on the interactive interface.

The instruction parameters input by the user on the interactive interface comprise instruction parameters of rotation, enlargement, reduction, translation or isosurface threshold value input by the user through a keyboard and/or a mouse.

And the fragment shader calculates the color value of the primitive according to the primitive information calculation result by adopting a Phong model.

The present invention also provides a GPU-based three-dimensional image data rendering apparatus, comprising:

one or more processors provided with a GPU;

a memory for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the GPU-based three-dimensional image data rendering method as described above.

In the three-dimensional image data rendering method based on the GPU provided by the embodiment of the invention, in the three-dimensional image visualization processing process, a Vertex Shader (Vertex Shader), a Geometry Shader (Geometry Shader) and a Fragment Shader (Fragment Shader) which are written based on glsl (opengl Shading language) are adopted, that is, a programmable rendering pipeline is adopted to draw a three-dimensional image (such as a medical image) by using the computation capability of the GPU, so that the computation burden of a CPU is reduced, the implementation is simple, and parameters are adjustable, thereby improving the interaction performance with a user in the three-dimensional image reconstruction process, and also improving the image visualization effect and performance.

Drawings

Fig. 1 is a flowchart of a three-dimensional image data rendering method according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a three-dimensional image data rendering apparatus according to an embodiment of the present invention;

FIG. 3 is an exemplary illustration of a three-dimensional image obtained by the rendering method according to the present invention;

FIG. 4 is an exemplary illustration of a three-dimensional image for FIG. 3 executing rotation instructions in accordance with the method of the present invention;

FIG. 5 is an exemplary illustration of a three-dimensional image for which the method of FIG. 3 performs a zoom-in instruction in accordance with the present invention;

FIG. 6 is an exemplary illustration of a three-dimensional image for which a zoom-out instruction is executed for FIG. 3 in accordance with the method of the present invention;

FIG. 7 is an exemplary illustration of a three-dimensional image for which the method of FIG. 3 performs the altering of the iso-surface threshold in accordance with the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention are described in detail below with reference to the accompanying drawings. Examples of these preferred embodiments are illustrated in the accompanying drawings. The embodiments of the invention shown in the drawings and described in accordance with the drawings are exemplary only, and the invention is not limited to these embodiments.

It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and/or processing steps closely related to the scheme according to the present invention are shown in the drawings, and other details not so relevant to the present invention are omitted.

The embodiment provides a three-dimensional image data rendering method, and referring to fig. 1, the method includes the steps of:

and S10, acquiring the three-dimensional image data, calculating vertex data, and setting a vertex group object and a vertex buffer object.

In this embodiment, a front-end processor of a Graphics Processing Unit (GPU) acquires three-dimensional image data by using a Marching Cubes algorithm, and specifically includes the following steps:

(1) and reading the area information of the whole area corresponding to the object to be processed into the three-dimensional array.

(2) Extracting a voxel from the three-dimensional array to become a current voxel and acquiring vertex information of the voxel; e.g., 8 vertex values, coordinates, etc.

(3) And comparing the value of the vertex of the current voxel with the value of the given isosurface to obtain the state of the voxel.

(4) And finding out the voxel edge intersected with the isosurface according to the state index of the current voxel, and calculating the position coordinates of each intersection point by adopting a linear interpolation method.

(5) And calculating a plane passing through each edge intersection point in the volume element, and obtaining an integral patch by taking the plane normal direction and each edge intersection point as a vertex to obtain integral three-dimensional image data.

After the three-dimensional image data is input into the GPU, the corresponding functional module in the GPU calculates and obtains Vertex data, and sets a Vertex Array Object (VAO) and a Vertex Buffer Object (VBO). Vertex data is an attribute that includes location coordinates, normals, colors, etc. The vertex buffer object VBO is a memory buffer area created in the graphics card storage space and is used to store various types of attribute information of a vertex, such as vertex coordinates, vertex normal vectors, vertex color data, and the like. During rendering, various types of attribute data of the vertex can be directly taken out from the VBO, and the processing efficiency is higher because the VBO is in a video memory rather than a memory. The VAO is a state binding that holds all of the vertex data attributes, storing the format of the vertex data and references to VBO objects needed for the vertex data. The VAO does not store related attribute data of the vertex, the information is stored in the VBO, the VAO is equivalent to the reference of a plurality of VBOs, and the VBOs are combined together to be uniformly managed as an object. All VBO configurations after execution of a VAO binding are part of this VAO object, which can be said to be a binding to vertex attribute information, and a VAO is a binding to many VBOs.

S20, edit Vertex Shader (Vertex Shader), Geometry Shader (Geometry Shader), and Fragment Shader (Fragment Shader) based on glsl (opengl Shading) and load in GPU.

And S30, reading vertex data by a vertex shader in the GPU and sending the vertex data to a geometry shader.

In the embodiment of the invention, the GPU is also provided with a Primitive assembling (primative Assembly) module, the vertex data read by the vertex shader is firstly sent to the Primitive assembling module, the Primitive assembling module assembles the vertex data into a Primitive with a specified shape according to the vertex data and then sends the Primitive to the geometry shader, and the geometry shader calculates the Primitive information according to the vertex data in the form of the Primitive. Specifically, the primitives of the specified shape are points, lines, or triangles.

And S40, performing primitive information calculation by a geometric shader in the GPU according to the vertex data, wherein the primitive information calculation comprises vertex coordinate space conversion, normal vector space conversion and illumination calculation.

Specifically, the geometry shader calculates primitive information according to vertex data and instruction parameters input by a user on the interactive interface. The instruction parameters input by the user on the interactive interface comprise instruction parameters of rotation, enlargement, reduction, translation or isosurface threshold value input by the user through a keyboard and/or a mouse.

And S50, calculating the color value of the primitive according to the calculation result of the primitive information by the fragment shader in the GPU, binding the vertex group object VAO, and writing the processed primitive information into a display card buffer area.

Specifically, the fragment shader calculates color values by adopting a Phong model according to received vertex coordinates and normal vectors and color information and camera position information transmitted by a main program, and improves the illumination real effect by adopting a flashlight model according to the camera position.

And S60, performing visual expression according to the graphic element information in the display card buffer area.

The present embodiment also provides a three-dimensional image data rendering apparatus, as shown in fig. 2, the three-dimensional image data rendering apparatus including: the system comprises a processor 10, a memory 20, an input device 30 and an output device 40, wherein the processor 10 is provided with a CPU and a GPU. The number of processors 10 in the three-dimensional image data rendering device may be one or more, and one processor 10 is taken as an example in fig. 2; the processor 10, the memory 20, the input device 30, and the output device 40 in the three-dimensional image data rendering apparatus may be connected by a bus or other means.

The memory 20, which is a computer-readable storage medium, may be used to store software programs, computer-executable programs, and modules. The processor 10 executes various functional applications of the apparatus and data processing, i.e., implements the three-dimensional image data rendering method in the foregoing embodiment of the present invention, by executing software programs, instructions, and modules stored in the memory 20. The input device 30 may be used to receive image data, input numeric or character information, and generate key signal inputs related to user settings and function control of the apparatus. The output device 40 may include a display device such as a display screen, for example, to display images.

Fig. 3 is an exemplary illustration of a rendering method apparatus according to the above embodiment of the present invention obtaining a three-dimensional image; FIG. 4 is an exemplary illustration of a three-dimensional image for FIG. 3 executing rotation instructions in accordance with the method of the present invention; FIG. 5 is an exemplary illustration of a three-dimensional image for which the method of FIG. 3 performs a zoom-in instruction in accordance with the present invention; FIG. 6 is an exemplary illustration of a three-dimensional image for which a zoom-out instruction is executed for FIG. 3 in accordance with the method of the present invention; FIG. 7 is an exemplary illustration of a three-dimensional image for which the method of FIG. 3 performs the altering of the iso-surface threshold in accordance with the present invention. Referring to fig. 3 to 6, in terms of an interactive function, a user may rotate an object by dragging a mouse and zoom in (zoom out) and zoom out (zoom out) of a camera (viewpoint) by a keyboard interaction. Comparing fig. 3 and fig. 7, both render the same individual data, but because the rendered contents (iso-surfaces) are different due to the different threshold values, the user can adjust the threshold values at any time and observe the change of the rendering effect in real time, which are real-time interactions.

In summary, in the three-dimensional image data rendering method based on the GPU provided by the embodiments of the present invention, in the three-dimensional image visualization processing process, the vertex shader, the geometry shader, and the fragment shader written based on the GLSL are used, that is, the programmable rendering pipeline is used to draw the three-dimensional volume image by using the computation capability of the GPU, so that the computation burden of the CPU is reduced, the implementation is simple, and the parameters are adjustable, thereby improving the interaction performance with the user in the three-dimensional image reconstruction process, and also improving the image visualization effect and performance.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

It should be noted that the above-mentioned embodiments are only for illustrating the technical concept and features of the present invention, and the purpose of the present invention is to enable those skilled in the art to understand the content of the present invention and implement the present invention, and not to limit the protection scope of the present invention. All equivalent changes and modifications made according to the spirit of the present invention should be covered within the protection scope of the present invention.

Claims (8)

Translated fromChinese

1.一种基于GPU的三维图像数据渲染方法，其特征在于，包括：1. a three-dimensional image data rendering method based on GPU, is characterized in that, comprises:获取三维图像数据并计算顶点数据，设置顶点数组对象和顶点缓冲对象；Obtain 3D image data and calculate vertex data, set vertex array object and vertex buffer object;基于GLSL编辑顶点着色器、几何着色器和片元着色器并加载在GPU中；Edit vertex shaders, geometry shaders and fragment shaders based on GLSL and load them in the GPU;在GPU中由顶点着色器读入顶点数据并发送给几何着色器；The vertex data is read in by the vertex shader in the GPU and sent to the geometry shader;在GPU中由几何着色器根据顶点数据进行图元信息计算，所述图元信息计算包括顶点坐标空间转换、法向量空间转换、光照计算；In the GPU, the geometry shader performs primitive information calculation according to the vertex data, and the primitive information calculation includes vertex coordinate space conversion, normal vector space conversion, and illumination calculation;在GPU中由片元着色器根据图元信息计算结果计算图元的颜色值并绑定顶点数组对象，将处理后的图元信息写入显卡缓存区；In the GPU, the fragment shader calculates the color value of the primitive according to the calculation result of the primitive information, binds the vertex array object, and writes the processed primitive information into the graphics card buffer;根据显卡缓存区中的图元信息进行可视化表达。Visual representation is made based on the primitive information in the graphics card buffer.2.根据权利要求1所述的基于GPU的三维图像数据渲染方法，其特征在于，所述获取三维图像数据是采用Marching Cubes算法获取，包括：2. GPU-based three-dimensional image data rendering method according to claim 1, is characterized in that, described acquiring three-dimensional image data is to adopt Marching Cubes algorithm to acquire, comprising:将待处理对象对应的整体区域的区域信息读入三维数组；Read the area information of the overall area corresponding to the object to be processed into a three-dimensional array;从三维数组中提取一个体元，成为当前体元，同时获取该体元顶点信息；Extract a voxel from the three-dimensional array to become the current voxel, and obtain the vertex information of the voxel at the same time;将当前体元的顶点的值与给定的等值面的值进行比较，得到该体元的状态；Compare the value of the vertex of the current voxel with the value of the given isosurface to get the state of the voxel;根据当前体元的状态索引，找出与等值面相交的体元棱边，并采用线性插值的方法，计算出各个交点的位置坐标；According to the state index of the current voxel, find the edge of the voxel that intersects with the isosurface, and use the method of linear interpolation to calculate the position coordinates of each intersection point;计算通过该体元中的各棱边交点的平面，以该平面法向方向、各棱边交点作为顶点，得到整体面片，获取整体三维图像数据。Calculate the plane passing through the intersection of each edge in the voxel, take the normal direction of the plane and the intersection of each edge as vertices to obtain an overall patch, and obtain overall 3D image data.3.根据权利要求1所述的基于GPU的三维图像数据渲染方法，其特征在于，GPU中还设置有图元装配模块，由顶点着色器读入顶点数据先发送至图元装配模块，由图元装配模块根据顶点数据装配成指定形状的图元再发送至几何着色器，几何着色器根据图元形式的顶点数据进行图元信息计算。3. The three-dimensional image data rendering method based on GPU according to claim 1, is characterized in that, in GPU, also be provided with primitive assembly module, read in vertex data by vertex shader and send to primitive assembly module first, by graph The meta assembly module assembles the primitives of the specified shape according to the vertex data and sends them to the geometry shader, and the geometry shader calculates primitive information according to the vertex data in the form of primitives.4.根据权利要求3所述的基于GPU的三维图像数据渲染方法，其特征在于，所述指定形状的图元为点、线或三角形。4 . The GPU-based three-dimensional image data rendering method according to claim 3 , wherein the primitives of the specified shape are points, lines or triangles. 5 .5.根据权利要求1-4任一所述的基于GPU的三维图像数据渲染方法，其特征在于，几何着色器根据顶点数据以及用户在交互界面输入的指令参数进行图元信息计算。5. The GPU-based three-dimensional image data rendering method according to any one of claims 1-4, wherein the geometry shader performs primitive information calculation according to vertex data and instruction parameters input by a user in an interactive interface.6.根据权利要求5所述的基于GPU的三维图像数据渲染方法，其特征在于，所述用户在交互界面输入的指令参数包括用户通过键盘和/或鼠标输入的旋转、放大、缩小、平移或等值面阈值的指令参数。6. The GPU-based three-dimensional image data rendering method according to claim 5, wherein the instruction parameters input by the user in the interactive interface include rotation, zoom-in, zoom-out, translation or Command parameter for isosurface thresholds.7.根据权利要求1所述的基于GPU的三维图像数据渲染方法，其特征在于，片元着色器采用Phong模型根据图元信息计算结果计算图元的颜色值。7 . The GPU-based three-dimensional image data rendering method according to claim 1 , wherein the fragment shader uses the Phong model to calculate the color value of the primitive according to the calculation result of the primitive information. 8 .8.一种基于GPU的三维图像数据渲染设备，其特征在于，包括：8. A GPU-based three-dimensional image data rendering device, comprising:一个或多个处理器，所述处理器设置有GPU；one or more processors provided with a GPU;存储器，用于存储一个或多个程序；memory for storing one or more programs;当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-7任一所述的基于GPU的三维图像数据渲染方法。When the one or more programs are executed by the one or more processors, the one or more processors implement the GPU-based three-dimensional image data rendering method according to any one of claims 1-7.

Images