Disclosure of Invention
In order to solve the technical problems, the invention provides a template matching implementation device and method based on an FPGA (field programmable gate array), which are used for template matching in a parallel computing mode, can quickly obtain results and meet the requirement of real-time monitoring. Therefore, the technical scheme of the invention is as follows:
a template matching implementation device based on FPGA comprises a memory and the FPGA;
the memory is used for storing an original image, the original image comprises n features to be matched, and each feature is respectively selected by frames to obtain n images of interest;
a ROM and N interesting region matching modules are arranged in the FPGA, and N is more than or equal to N; the n interesting region images are respectively stored in an interesting region matching module;
storing template data in the ROM;
a plurality of double-port BRAMs, a matching calculation module and a comparison module are arranged in each interest region matching module, and each double-port BRAM is connected with two matching calculation modules respectively; all the matching calculation modules are connected with the comparison module; the single frame of the interested region image is divided into a plurality of sections, the partial data of adjacent sections are the same, and each section of image is stored in a dual-port BRAM; the matching calculation module is connected with the ROM, can acquire template data therein, and is used for matching the data received from the BRAM with the template data to acquire a matching metric value, then comparing the matching metric value with a larger/smaller value in a previous group of comparison results, screening the larger/smaller value, preparing for comparison with a next result, and until a maximum/minimum matching metric value of the image and the template which are stored in the same dual-port BRAM and a corresponding matching point coordinate are acquired; outputting the single image as a matching measurement result of a single image in a single image of interest;
the comparison module compares the matching metric values of the segmented images in the single interested region image to obtain the maximum/minimum value, namely the optimal matching metric value of the single interested region and the template image, and simultaneously obtains the corresponding matching point coordinates of the single interested region and the template image.
After n interesting region images in the same image are respectively matched, n results can be obtained.
Further, the memory has two separate storage spaces, and the original images are stored in the two storage spaces in an alternating manner in sequence.
Further, the method for dividing the single frame of the region of interest image into a plurality of segments comprises the following steps:
a first stage: line 1 to line 1
A row;
and a second stage: first, the
Go to
A row;
a third stage: first, theGo toA row;
……
an M section:moving to row B;
wherein: b is the total line number of the interested region image; b' is the total line number of the template image; m is the total number of segments into which the image of interest is divided;
is rounded up.
Further, the minimum value of the segment number M of the interesting image division is determined according to the following formula:
wherein: the sliding window range refers to the area of the template image which can freely move on the image of interest in the image matching process; the preset template matching time refers to the allowed time for completing matching of n interesting region images on the same frame of original image;is rounded up.
A template matching implementation method based on FPGA comprises the following steps:
1) performing frame selection on n features needing to be matched on an original image to obtain n interesting region images; storing the original image in a memory;
2) the n interesting region images are respectively stored in an interesting region matching module in the FPGA, and the interesting region matching module comprises a plurality of dual-port BRAMs, a matching calculation module and a comparison module; the storage mode of the interested region image is as follows: sequentially dividing each image of interest into a plurality of sections, wherein the data of the two adjacent sections are the same, and each section of data is independently stored in a dual-port BRAM;
3) the single dual-port BRAM is connected with two matching calculation modules, data are respectively transmitted to the matching calculation modules for matching calculation, the matching calculation modules call template data from the ROM to be matched with the received image data of the region of interest, a matching metric value is obtained, the matching metric value is compared with a larger value/a smaller value in a previous group of comparison results, a larger value/a smaller value is screened, and the matching metric value is prepared to be compared with a next obtained result until the maximum matching metric value/the minimum matching metric value of the image and the template which are stored in the same dual-port BRAM and the corresponding matching point coordinate of the maximum matching metric value/the minimum matching metric value are obtained; outputting the single image as a matching measurement result of a single image in a single image of interest;
4) the comparison module compares the matching metric values of the segmented images in the single interested region image to obtain the maximum/minimum value, namely the optimal matching metric value of the single interested region and the template image, and simultaneously obtains the corresponding matching point coordinates of the single interested region and the template image.
Further, the memory has two separate storage spaces, and the original images are stored in the two storage spaces in an alternating manner in sequence.
Further, the method for sequentially dividing each region of interest image into a plurality of segments in the step 2) comprises the following steps:
a first stage: line 1 to line 1
A row;
and a second stage: first, the
Go to
A row;
a third stage: first, the
Go to
A row;
……
an M section:moving to row B;
wherein: b is the total line number of the interested region image; b' is the total line number of the template image; m is the total number of segments into which the image of interest is divided;is rounded up.
Further, the minimum value of the segment number M of the interesting image division is determined according to the following formula:
wherein: the sliding window range refers to the area of the template image which can freely move on the image of interest in the image matching process; the preset template matching time refers to the allowed time for completing matching of n interesting region images on the same frame of original image;is rounded up.
At present, the following six conventional template matching metric value calculation methods can be adopted for calculation by using the method/device provided by the invention. Naturally, the method provided by the present invention is not limited to these six calculation methods, and may be applied to other calculation methods.
1. Squared error matching
Wherein, R (x, y) is a matching value, T represents a template drawing, I represents an area currently covered by the template drawing, and x 'and y' are values of x and y coordinates in the template drawing T respectively. (the same letter meaning)
2. Standard squared error matching
3. Correlation matching
4. Standard correlation matching
5. Correlation matching
Wherein:
T'(x',y')=T(x',y')-1/(w·h)·∑x”,y”T'(x”,y”)
I'(x+x',y+y')=I(x+x',y+y')-1/(w·h)·∑x”,y”I(x+x”,y+y”)
6. standard correlation matching
The device and the method for realizing template matching based on the FPGA adopt a parallel computing mode to carry out template matching, can quickly obtain results, reduce the power consumption of equipment, and can adjust parameters according to the requirement of computing time to meet the requirement of real-time monitoring.
Detailed Description
The technical solution of the present invention is described in detail below with reference to the accompanying drawings and the detailed description.
A template matching implementation device based on FPGA comprises a memory and the FPGA;
the memory is used for storing an original image, the original image comprises n features to be matched, and each feature is respectively selected by frames to obtain n images of interest (as shown in figure 1);
a ROM and N interesting region matching modules are arranged in the FPGA, and N is more than or equal to N; the n interesting area images are respectively stored in an interesting area matching module;
storing template data in the ROM;
a plurality of double-port BRAMs, a matching calculation module and a comparison module are arranged in the region of interest matching module, and each double-port BRAM is connected with two matching calculation modules respectively; all the matching calculation modules are connected with the comparison module; the single frame interesting region image is divided into a plurality of sections, the partial data of the adjacent sections are the same, and each section of image is stored in a dual-port BRAM; specifically, the method comprises the following steps: the single frame interesting region image is divided into M sections by adopting the following method:
a first stage: line 1 to line 1A row;
and a second stage: first, the
Go to
A row;
a third stage: first, the
Go to
A row;
……
an M section:
moving to row B;
wherein: b is the total line number of the interested region image; b' is the total line number of the template image; m is the total number of segments into which the image of interest is divided;
is to round up upwards;
determining the minimum value of the segment number M of the interesting image division according to the following formula:
wherein: the sliding window range refers to the area of the template image which can freely move on the image of interest in the image matching process; the preset template matching time refers to the allowed time for completing matching of n interesting region images on the same frame of original image;
is rounded up.
The matching calculation module is connected with the ROM, can obtain template data therein, and is used for matching the data received from the BRAM with the template data to obtain a matching metric value, then comparing the matching metric value with a larger/smaller value in a previous group of comparison results, screening the larger/smaller value, preparing to compare with a next obtained result until obtaining a maximum/minimum matching metric value of the image and the template which are stored in the same dual-port BRAM and a corresponding matching point coordinate; outputting the single image as a matching measurement result of a single image in a single image of interest;
the comparison module compares the matching metric values of the segmented images in the single interested region image to obtain the maximum/minimum value, namely the optimal matching metric value of the single interested region and the template image, and simultaneously obtains the corresponding matching point coordinates of the single interested region and the template image.
After n interesting region images in the same image are respectively matched, n results can be obtained.
In order to accelerate the calling speed of the picture, two separate storage spaces are stored in the memory, and the original images are stored in the two storage spaces in an alternating mode in sequence. The memory can be divided into two blocks or one block.
A template matching implementation method based on FPGA comprises the following steps:
1) performing frame selection on n features needing to be matched on an original image to obtain n interesting region images; storing the original image in a memory;
2) the n interesting area images are respectively stored in an interesting area matching module in the FPGA, and the interesting area matching module comprises a plurality of dual-port BRAMs, a matching calculation module and a comparison module; the storage mode of the interested region image is as follows: sequentially dividing each interested region image into M sections, wherein the data of the two adjacent sections are the same, and each section of data is independently stored in a dual-port BRAM; determining the minimum value of the segment number of the interesting image division according to the following formula:
wherein: the sliding window range refers to the area of the template image which can freely move on the image of interest in the image matching process; the preset template matching time refers to the allowed time for completing matching of n interesting region images on the same frame of original image;
is rounded up.
Specifically, the method comprises the following steps: the method for sequentially dividing each interested region image into a plurality of sections comprises the following steps:
a first stage: line 1 to line 1
A row;
and a second stage: first, the
Go to
A row;
a third stage: first, the
Go to
A row;
……
an M section:
go to row B;
Wherein: b is the total line number of the interested region image; b' is the total line number of the template image; m is the total number of segments into which the image of interest is divided;
is to round up upwards; in addition to this division method, other division methods may be naturally selected as long as the storage condition is satisfied;
3) the single dual-port BRAM is connected with two matching calculation modules, data are respectively transmitted to the matching calculation modules for matching calculation, the matching calculation modules call template data from the ROM to be matched with the received image data of the region of interest, a matching metric value is obtained, the matching metric value is compared with a larger value/a smaller value in a previous group of comparison results, a larger value/a smaller value is screened, and the matching metric value is prepared to be compared with a next obtained result until the maximum matching metric value/the minimum matching metric value of the image and the template which are stored in the same dual-port BRAM and the corresponding matching point coordinate of the maximum matching metric value/the minimum matching metric value are obtained; outputting the single image as a matching measurement result of a single image in a single image of interest;
4) the comparison module compares the matching metric values of the segmented images in the single interested region image to obtain the maximum/minimum value, namely the optimal matching metric value of the single interested region and the template image, and simultaneously obtains the corresponding matching point coordinates of the single interested region and the template image.
In order to call the picture data in parallel, the internal memory has two separate storage spaces, which can be realized by two internal memories, or by dividing one internal memory into two areas, and the original image is stored in the two storage spaces in an alternating mode in sequence.
At present, the following six conventional template matching metric value calculation methods can be adopted for calculation by using the method/device provided by the invention. Naturally, the method provided by the present invention is not limited to these six calculation methods, and may be applied to other calculation methods.
1. Squared error matching
Wherein, R (x, y) is a matching value, T represents a template drawing, I represents an area currently covered by the template drawing, and x 'and y' are values of x and y coordinates in the template drawing T respectively. (the same letter meaning)
2. Standard squared error matching
3. Correlation matching
4. Standard correlation matching
5. Correlation matching
Wherein:
T'(x',y')=T(x',y')-1/(w·h)·∑x”,y”T'(x”,y”)
I'(x+x',y+y')=I(x+x',y+y')-1/(w·h)·∑x”,y”I(x+x”,y+y”)
6. standard correlation matching
The device and the method for realizing template matching based on the FPGA adopt a parallel computing mode to carry out template matching, can quickly obtain results, reduce the power consumption of equipment and meet the requirement of real-time monitoring.
The following compares the effects of the method of the present invention and the existing method with the same treatment object:
a template matching implementation method based on FPGA comprises the following steps:
1) 5 features needing to be matched are arranged on the original image; the original image is stored in the memory, 5 templates are matched with 5 features, the size of each template is 100 multiplied by 100, and the number of effective points is 10000.
2) The 5 interesting region images are respectively stored in an interesting region matching module in the FPGA, the size of the interesting region image is 299 multiplied by 299, and the range of the sliding window is 200 multiplied by 200. The storage mode of the interested region image is as follows: sequentially dividing each interested region image into four sections, wherein the data of the two adjacent sections are the same, and each section of data is independently stored in a dual-port BRAM;
specifically, the method comprises the following steps: the method for sequentially dividing each interested region image into four sections comprises the following steps:
a first stage: lines 1 through 149;
and a second stage: lines 51 through 199;
a third stage: lines 101 through 249;
a fourth stage: lines 151 through 299;
the same as the steps 3) and 4) above, calculating by adopting a 4 th standard correlation matching method; the running clock frequency of the FPGA is 200000000Hz, and the time required for matching 5 characteristics is as follows:
comparative example: under the condition that the template size and the sliding window range are the same, the consumed time is 10s when the dual-core Cortex A9 runs by using the same template matching algorithm. Description
Therefore, the template matching speed can be greatly improved by using the FPGA-based template matching method provided by the application.
The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. The foregoing description is not intended to be exhaustive or to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical application to enable others skilled in the art to make and use various exemplary embodiments of the invention and various alternatives and modifications thereof. It is intended that the scope of the invention be defined by the following claims and their equivalents.