CN114419576A - Traffic sign identification method and system based on category mapping - Google Patents
Traffic sign identification method and system based on category mappingDownload PDFInfo
- Publication number
- CN114419576A CN114419576ACN202111467675.3ACN202111467675ACN114419576ACN 114419576 ACN114419576 ACN 114419576ACN 202111467675 ACN202111467675 ACN 202111467675ACN 114419576 ACN114419576 ACN 114419576A
- Authority
- CN
- China
- Prior art keywords
- result
- branch
- traffic sign
- traffic
- speed limit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034methodMethods0.000titleclaimsabstractdescription45
- 238000013507mappingMethods0.000titleclaimsabstractdescription33
- 238000000605extractionMethods0.000claimsabstractdescription37
- 238000004590computer programMethods0.000claimsdescription18
- 239000003550markerSubstances0.000claims1
- 238000012549trainingMethods0.000abstractdescription8
- 238000001514detection methodMethods0.000abstractdescription6
- 230000011218segmentationEffects0.000abstractdescription2
- 238000004891communicationMethods0.000description7
- 238000010586diagramMethods0.000description4
- 230000009286beneficial effectEffects0.000description3
- 238000013528artificial neural networkMethods0.000description2
- 230000007547defectEffects0.000description2
- 230000000694effectsEffects0.000description2
- 230000003287optical effectEffects0.000description2
- 238000005457optimizationMethods0.000description2
- 238000010276constructionMethods0.000description1
- 238000012937correctionMethods0.000description1
- 230000006870functionEffects0.000description1
- 238000012986modificationMethods0.000description1
- 230000004048modificationEffects0.000description1
- 238000007781pre-processingMethods0.000description1
Images
Landscapes
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本发明涉及语义分割技术领域，尤其涉及一种基于类别映射的交通标志识别方法和系统。The present invention relates to the technical field of semantic segmentation, in particular to a method and system for identifying traffic signs based on category mapping.
背景技术Background technique
道路交通标志和标线时引导道路使用者有秩序使用道路，以促进道路行车安全，而在驾驶辅助系统中对交通标志的识别则可以不间断的为整车控制提供相应的帮助。比如禁止类标志可以帮助系统提前进行危险预判；警告类标志可以帮助系统提前进行在某些情况下进行提前避障处理；指示类标志可以帮助系统进行控制预处理，以确保行车遵循道路指示。故对于交通标志的正确识别及精准应用可以为驾驶辅助系统甚至自动驾驶提供更加完美的助攻。Road traffic signs and markings guide road users to use the road in an orderly manner to promote road driving safety, while the recognition of traffic signs in the driver assistance system can provide uninterrupted assistance for vehicle control. For example, prohibition signs can help the system to predict dangers in advance; warning signs can help the system to avoid obstacles in advance in some cases; indication signs can help the system to perform control preprocessing to ensure that traffic follows road instructions. Therefore, the correct recognition and precise application of traffic signs can provide more perfect assistance for driving assistance systems and even automatic driving.
然而，受限于车载终端的硬件成本、体积、环境等因素，自动驾驶或辅助驾驶场景下，汽车本地的硬件资源捉襟见肘，能够分配至交通标志识别任务的计算资源通常较为有限。However, limited by factors such as the hardware cost, size, and environment of the vehicle terminal, in the scenario of automatic driving or assisted driving, the local hardware resources of the car are stretched, and the computing resources that can be allocated to the task of traffic sign recognition are usually limited.
对于限速标志而言，计算资源的问题尤为突出，即由于限速类型和数值的多样性，现有的基于神经网络的汽车本地实时识别方法存在着计算资源需求高、识别速度慢的缺陷。For speed limit signs, the problem of computing resources is particularly prominent, that is, due to the diversity of speed limit types and values, the existing local real-time recognition methods based on neural networks have the defects of high computing resource requirements and slow recognition speed.
因此，如何提供一种高效、快速的交通标志识别方法和系统，成为了业内亟需解决的技术问题。Therefore, how to provide an efficient and fast traffic sign recognition method and system has become an urgent technical problem to be solved in the industry.
发明内容SUMMARY OF THE INVENTION
本发明提供一种基于类别映射的交通标志识别方法和系统，用以解决现有技术中计算资源需求高、识别速度慢的缺陷，实现高效、快速的交通标志识别。The invention provides a traffic sign recognition method and system based on class mapping, which is used to solve the defects of high computing resource demand and slow recognition speed in the prior art, and realize efficient and fast traffic sign recognition.
本发明提供一种基于类别映射的交通标志识别方法，应用于车载终端，包括：The present invention provides a traffic sign recognition method based on class mapping, which is applied to a vehicle-mounted terminal, including:
通过车载图像采集设备获取道路图像；Obtain road images through on-board image acquisition equipment;
将所述道路图像输入识别网络的特征提取层，得到提取结果；Inputting the road image into the feature extraction layer of the recognition network to obtain an extraction result;
将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支得到交通标志的第一结果、第二结果，并根据所述第一结果和第二结果得到交通标志识别结果；Input the extraction result into the decoder layer of the recognition network, obtain the first result and the second result of the traffic sign through the first branch and the second branch of the decoder layer, and obtain the first result and the second result of the traffic sign according to the first result and the second branch. As a result, the traffic sign recognition result is obtained;
所述第一结果为所述交通标志的类别；所述第二结果为所述交通标志的数值；the first result is the category of the traffic sign; the second result is the value of the traffic sign;
所述识别网络是基于样本和标签训练得到的。The recognition network is trained based on samples and labels.
根据本发明提供的一种基于类别映射的交通标志识别方法，所述将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支得到交通标志的第一结果、第二结果，并根据所述第一结果和第二结果得到交通标志识别结果的步骤包括：According to a traffic sign recognition method based on class mapping provided by the present invention, the extraction result is input into the decoder layer of the recognition network, and the first branch and the second branch of the decoder layer respectively obtain the traffic signs. The first result, the second result, and the steps of obtaining the traffic sign recognition result according to the first result and the second result include:
将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支、第三分支、第四分支得到交通标志的第一结果、第二结果、第三结果、第四结果，并根据所述第一结果、第二结果、第三结果以及第四结果得到交通标志识别结果；Input the extraction result into the decoder layer of the recognition network, and obtain the first result, the second result, the third result of the traffic sign through the first branch, the second branch, the third branch and the fourth branch of the decoder layer respectively. result, the fourth result, and obtain the traffic sign recognition result according to the first result, the second result, the third result and the fourth result;
所述第三结果为交通标志的宽高；所述第四结果为交通标志中心点坐标的偏移预测。The third result is the width and height of the traffic sign; the fourth result is the offset prediction of the coordinates of the center point of the traffic sign.
根据本发明提供的一种基于类别映射的交通标志识别方法：According to a traffic sign recognition method based on class mapping provided by the present invention:
所述第一分支包括针对路面最高限速类别、路面最低限速类别、路面解除限速类别、非路面最高限速类别、非路面最低限速类别以及非路面解除限速类别的6个通道；The first branch includes 6 channels for the maximum speed limit category on the road, the minimum speed limit category on the road, the speed limit released on the road category, the maximum speed limit category off the road, the minimum speed limit category off the road, and the speed limit lifted off the road category;
所述第二分支包括针对交通数值的设定数量的通道；the second branch includes a set number of channels for traffic values;
所述第三分支包括针对交通标志宽、高的2个通道；The third branch includes 2 passages for the width and height of the traffic sign;
所述第四分支包括针对交通标志中心点在横、纵坐标轴方向偏移量的2个通道。The fourth branch includes two channels for the offset of the center point of the traffic sign in the direction of the horizontal and vertical axes.
根据本发明提供的一种基于类别映射的交通标志识别方法，所述交通数值是基于道路环境信息确定的。According to a traffic sign identification method based on class mapping provided by the present invention, the traffic value is determined based on road environment information.
根据本发明提供的一种基于类别映射的交通标志识别方法，所述第一分支包括标志位置子分支、交通类型子分支；所述第一结果包括通过所述标志位置子分支得到的标志位置结果和通过所述交通类型子分支得到的交通类型结果。According to a traffic sign recognition method based on class mapping provided by the present invention, the first branch includes a sign position sub-branch and a traffic type sub-branch; the first result includes a sign position result obtained through the sign position sub-branch and the traffic type results obtained through the traffic type sub-branch.
根据本发明提供的一种基于类别映射的交通标志识别方法：According to a traffic sign recognition method based on class mapping provided by the present invention:
所述标志位置子分支包括针对路面交通标志和非路面交通标志的2个通道；The sign position sub-branch includes 2 passages for road traffic signs and non-road traffic signs;
所述交通类型子分支包括针对最高限速类别、最低限速类别以及解除限速类别的3个通道。The traffic type sub-branch includes 3 channels for the highest speed limit category, the lowest speed limit category, and the release speed limit category.
本发明还提供一种基于类别映射的交通标志识别系统，部署于车载终端，包括：The present invention also provides a traffic sign recognition system based on class mapping, deployed in the vehicle terminal, including:
获取模块，用于通过车载图像采集设备获取道路图像；an acquisition module, used to acquire road images through vehicle-mounted image acquisition equipment;
特征提取模块，用于将所述道路图像输入识别网络的特征提取层，得到提取结果；a feature extraction module, for inputting the road image into the feature extraction layer of the recognition network to obtain an extraction result;
解码模块，用于将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支得到交通标志的第一结果、第二结果，并根据所述第一结果和第二结果得到交通标志识别结果；The decoding module is used to input the extraction result into the decoder layer of the recognition network, obtain the first result and the second result of the traffic sign through the first branch and the second branch of the decoder layer, and obtain the first result and the second result of the traffic sign according to the first branch and the second branch of the decoder layer. The first result and the second result obtain the traffic sign recognition result;
所述第一结果为所述交通标志的类别；所述第二结果为所述交通标志的数值；the first result is the category of the traffic sign; the second result is the value of the traffic sign;
所述识别网络是基于样本和标签训练得到的。The recognition network is trained based on samples and labels.
本发明还提供一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时实现如上述任一种所述基于类别映射的交通标志识别方法的步骤。The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the program, the class-based mapping as described in any of the above is implemented. The steps of the traffic sign recognition method.
本发明还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现如上述任一种所述基于类别映射的交通标志识别方法的步骤。The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of any one of the above-mentioned traffic sign recognition methods based on class mapping.
本发明还提供一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现如上述任一种所述基于类别映射的交通标志识别方法的步骤。The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned traffic sign recognition methods based on class mapping.
本发明提供的基于类别映射的交通标志识别方法和系统，通过解码器层的分支网络分解交通标志识别任务，使得第一分支、第二分支分别专注于交通标志的类别识别和数值识别，从而将可能作为识别结果的检测类数量由类别数量与数值数量的乘积简化为类别数量与数值数量的和，一方面简化了网络结构、减少了计算资源需求，另一方面利用针对性的训练提升了网络的识别精度和速度。The traffic sign recognition method and system based on class mapping provided by the present invention decomposes the traffic sign recognition task through the branch network of the decoder layer, so that the first branch and the second branch focus on the class recognition and numerical recognition of the traffic sign respectively, so that the The number of detection classes that may be used as a recognition result is simplified from the product of the number of categories and the number of values to the sum of the number of categories and the number of values. On the one hand, the network structure is simplified and the computing resource requirements are reduced. recognition accuracy and speed.
附图说明Description of drawings
为了更清楚地说明本发明或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are the For some embodiments of the invention, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.
图1是本发明提供的基于类别映射的交通标志识别方法的流程示意图；1 is a schematic flowchart of a method for identifying traffic signs based on class mapping provided by the present invention;
图2是本发明实施例提供的识别网络的结构示意图；2 is a schematic structural diagram of an identification network provided by an embodiment of the present invention;
图3是本发明提供的基于类别映射的交通标志识别系统的结构示意图；3 is a schematic structural diagram of a class mapping-based traffic sign recognition system provided by the present invention;
图4是本发明提供的电子设备的结构示意图。FIG. 4 is a schematic structural diagram of an electronic device provided by the present invention.
附图标记：Reference number:
1：获取模块； 2：特征提取模块； 3：解码模块；1: Acquisition module; 2: Feature extraction module; 3: Decoding module;
410：处理器； 420：通信接口； 430：存储器；410: processor; 420: communication interface; 430: memory;
440：通信总线。440: Communication bus.
具体实施方式Detailed ways
为使本发明的目的、技术方案和优点更加清楚，下面将结合本发明中的附图，对本发明中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention. , not all examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
下面结合图1、图2描述本发明的基于类别映射的交通标志识别方法。The traffic sign recognition method based on class mapping of the present invention will be described below with reference to FIG. 1 and FIG. 2 .
如图1所示，本发明实施例提供一种基于类别映射的交通标志识别方法，应用于车载终端，包括：As shown in FIG. 1 , an embodiment of the present invention provides a traffic sign recognition method based on class mapping, which is applied to a vehicle-mounted terminal, including:
步骤102，通过车载图像采集设备获取道路图像；
步骤104，将所述道路图像输入识别网络的特征提取层，得到提取结果；
步骤106，将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支得到交通标志的第一结果、第二结果，并根据所述第一结果和第二结果得到交通标志识别结果；Step 106: Input the extraction result into the decoder layer of the recognition network, obtain the first result and the second result of the traffic sign through the first branch and the second branch of the decoder layer respectively, and obtain the first result and the second result of the traffic sign according to the first result. and the second result to obtain the traffic sign recognition result;
所述第一结果为所述交通标志的类别；所述第二结果为所述交通标志的数值；the first result is the category of the traffic sign; the second result is the value of the traffic sign;
所述识别网络是基于样本和标签训练得到的。The recognition network is trained based on samples and labels.
本实施例中，所述解码器层的第一分支用于对交通标志的类别进行识别，如禁止机动车驶入标志、注意合流标志、施工标志、最高限速标志、解除限速标志、最低限速标志、车道行驶方向标志等；In this embodiment, the first branch of the decoder layer is used to identify the types of traffic signs, such as prohibiting motor vehicles from entering signs, attention to merging signs, construction signs, maximum speed limit signs, release speed limit signs, minimum Speed limit signs, lane driving direction signs, etc.;
所述解码器层的第二分支用于对交通标志的数值进行识别；所述数值包括交通标志的数字值(如限速标志的速度值、出口编号标志的编号值、百米牌的数值、里程标志的里程数值等)和/或预设的定义数值(如定义车道行驶方向标志直行为-1、车道行驶方向标志左转加调头为-2、改道标志向左为-3等)。The second branch of the decoder layer is used to identify the numerical value of the traffic sign; the numerical value includes the numerical value of the traffic sign (such as the speed value of the speed limit sign, the number value of the exit number sign, the value of the 100m sign, The mileage value of the mileage sign, etc.) and/or the preset definition value (such as defining the lane driving direction sign to go straight as -1, the lane driving direction sign to turn left plus U-turn to be -2, to the left of the diversion sign to be -3, etc.).
在一个优选的实施方式中，所述第一结果有多个，且每个第一结果均由唯一确定的交通标志类别及其概率构成；所述第二结果有多个，且每个第二结果均由唯一确定的数值及其概率构成；所述交通标志识别结果是通过概率最高的第一结果和概率最高的第二结果得到的唯一结果，或者通过概率大于阈值的第一结果和概率大于阈值的第二结果组合得到多个结果。In a preferred embodiment, there are multiple first results, and each first result is composed of a uniquely determined traffic sign category and its probability; there are multiple second results, and each second result is The result is composed of a uniquely determined value and its probability; the traffic sign recognition result is the only result obtained through the first result with the highest probability and the second result with the highest probability, or the first result with the probability greater than the threshold and the probability greater than The second result of the threshold is combined to obtain multiple results.
在一个优选的实施方式中，所述识别网络用于识别限速标志；所述第一分支用于识别限速标志的限速类型；所述第二分支用于识别限速标志的限速值。In a preferred embodiment, the identification network is used to identify the speed limit sign; the first branch is used to identify the speed limit type of the speed limit sign; the second branch is used to identify the speed limit value of the speed limit sign .
在不考虑道路类型的情况下，所述限速类型包括路面最高限速类别、路面最低限速类别、路面解除限速类别、非路面最高限速类别、非路面最低限速类别以及非路面解除限速类别这6个类别；所述限速数值包括5、10、15、20、25、30、35、40、45、50、55、60、65、70、75、80、85、90、95、100、105、110、115、120这24个类别；Without considering the type of road, the speed limit type includes maximum speed limit category on the road, minimum speed limit category on the road, speed limit released on the road category, maximum speed limit category off the road, minimum speed limit category off the road and off the road release The 6 categories of speed limit categories; the speed limit values include 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 24 categories of 95, 100, 105, 110, 115, 120;
故总类别数为6与24的乘积，即144种，不考虑本实施例方法的优化，这144种类别的限速标志需要至少144个通道进行识别，所需计算资源较高，且训练效果较差(事实上，针对限速标志和其它更多交通标志的识别网络中，训练效果差的问题更为突出)；通过本实施方式的优化，基于第一分支和第二分支的运算，交通标志的识别结果简化为30种，只需通过30个通道即可完成识别。Therefore, the total number of categories is the product of 6 and 24, that is, 144 types. Regardless of the optimization of the method in this embodiment, these 144 types of speed limit signs require at least 144 channels to identify, which requires high computing resources and has a good training effect. (In fact, in the recognition network for speed limit signs and other more traffic signs, the problem of poor training effect is more prominent); through the optimization of this embodiment, based on the operations of the first branch and the second branch, the traffic The identification results of the logo are simplified to 30 types, and the identification can be completed only through 30 channels.
进一步地，在考虑道路类型的情况下，第二分支的通道数可以随之减少，例如对于城市快速路，常见的限速值为80、60、40，则针对城市快速路的识别网络中，解码器层的第二分支通道数可以设置为3，并分别针对80、60、40进行识别。Further, in the case of considering the road type, the number of channels of the second branch can be reduced accordingly. For example, for urban expressways, the common speed limit values are 80, 60, and 40. In the identification network for urban expressways, The number of second branch channels of the decoder layer can be set to 3 and identified for 80, 60, and 40 respectively.
上述针对限速标志的实施方式说明是为了从量化的角度更好的说明本实施例及其有益效果，并不构成对本发明保护范围的限制。The above description of the implementation of the speed limit sign is for the purpose of better explaining the present embodiment and its beneficial effects from a quantitative point of view, and does not constitute a limitation on the protection scope of the present invention.
本实施例的有益效果在于：The beneficial effects of this embodiment are:
通过解码器层的分支网络分解交通标志识别任务，使得第一分支、第二分支分别专注于交通标志的类别识别和数值识别，从而将可能作为识别结果的检测类数量由类别数量与数值数量的乘积简化为类别数量与数值数量的和，一方面简化了网络结构、减少了计算资源需求，另一方面利用针对性的训练提升了网络的识别精度和速度。The traffic sign recognition task is decomposed by the branch network of the decoder layer, so that the first branch and the second branch focus on the classification and numerical recognition of traffic signs respectively, so that the number of detection classes that may be used as recognition results is determined by the number of categories and the number of values. The product is simplified to the sum of the number of categories and the number of values. On the one hand, the network structure is simplified and the computing resource requirements are reduced. On the other hand, targeted training is used to improve the recognition accuracy and speed of the network.
根据上述实施例，在本实施例中：According to the above-mentioned embodiment, in this embodiment:
所述将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支得到交通标志的第一结果、第二结果，并根据所述第一结果和第二结果得到交通标志识别结果的步骤包括：Inputting the extraction result into the decoder layer of the recognition network, respectively obtaining the first result and the second result of the traffic sign through the first branch and the second branch of the decoder layer, and obtaining the first result and the second result of the traffic sign according to the first result and The step of obtaining the traffic sign recognition result in the second result includes:
将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支、第三分支、第四分支得到交通标志的第一结果、第二结果、第三结果、第四结果，并根据所述第一结果、第二结果、第三结果以及第四结果得到交通标志识别结果；Input the extraction result into the decoder layer of the recognition network, and obtain the first result, the second result, the third result of the traffic sign through the first branch, the second branch, the third branch and the fourth branch of the decoder layer respectively. result, the fourth result, and obtain the traffic sign recognition result according to the first result, the second result, the third result and the fourth result;
所述第三结果为交通标志的宽高；所述第四结果为交通标志中心点坐标的偏移预测。The third result is the width and height of the traffic sign; the fourth result is the offset prediction of the coordinates of the center point of the traffic sign.
所述第一分支包括针对路面最高限速类别、路面最低限速类别、路面解除限速类别、非路面最高限速类别、非路面最低限速类别以及非路面解除限速类别的6个通道；The first branch includes 6 channels for the maximum speed limit category on the road, the minimum speed limit category on the road, the speed limit released on the road category, the maximum speed limit category off the road, the minimum speed limit category off the road, and the speed limit lifted off the road category;
所述第二分支包括针对交通数值的设定数量的通道；the second branch includes a set number of channels for traffic values;
所述第三分支包括针对交通标志宽、高的2个通道；The third branch includes 2 passages for the width and height of the traffic sign;
所述第四分支包括针对交通标志中心点在横、纵坐标轴方向偏移量的2个通道。The fourth branch includes two channels for the offset of the center point of the traffic sign in the direction of the horizontal and vertical axes.
所述交通数值是基于道路环境信息确定的。The traffic value is determined based on road environment information.
本实施例中，所述第四分支的设置是为了解决由于输入图片像素的离散性和二进制计算机系统中近似取整造成的，识别结果的交通标志中心点不准确，可能产生偏差的问题。In this embodiment, the setting of the fourth branch is to solve the problem that the center point of the traffic sign in the recognition result is inaccurate due to the discreteness of the input picture pixels and the approximate rounding in the binary computer system, which may cause deviation.
在一个优选的实施方式中，所述第一分支和/或第二分支的输出，即第一结果和/或第二结果为概率图形式的结果；所述概率图是基于通道生成的，包括识别结果为通道针对交通标志中心点的概率分布；例如，对于第一分支的非路面解除限速通道，其输出的第一结果包括离散或连续分布的概率点，所述概率点包括点坐标和概率值。所述点坐标是指交通标志中心点，所述概率值是指交通标志中心点位于所述点坐标处、且该交通标志为非路面解除限速标志的概率。In a preferred embodiment, the output of the first branch and/or the second branch, that is, the first result and/or the second result is a result in the form of a probability map; the probability map is generated based on channels, including The recognition result is the probability distribution of the channel for the center point of the traffic sign; for example, for the non-road speed-limited channel of the first branch, the output first result includes discrete or continuous distribution probability points, and the probability points include point coordinates and probability value. The point coordinates refer to the center point of the traffic sign, and the probability value refers to the probability that the center point of the traffic sign is located at the coordinates of the point, and the traffic sign is a non-road speed limit release sign.
进一步地，最终得到的交通标志识别结果中：Further, in the final traffic sign recognition result:
交通标志的中心点是以第一分支和/或第二分支的输出，即第一结果和/或第二结果中，概率值最高的概率点对应的点坐标为基础，结合第四分支的输出，即第四结果中，交通标志中心点坐标的偏移预测修正后得到的；The center point of the traffic sign is based on the output of the first branch and/or the second branch, that is, the point coordinates corresponding to the probability point with the highest probability value in the first result and/or the second result, combined with the output of the fourth branch , that is, in the fourth result, the offset prediction of the coordinates of the center point of the traffic sign is obtained after correction;
交通标志(或者交通标志检测框)的宽高是由第三分支的输出，即第三结果确定的；The width and height of the traffic sign (or the traffic sign detection frame) is determined by the output of the third branch, that is, the third result;
交通标志的类型是第一分支的输出，即第一结果中，概率值最高的概率点对应的通道类型；The type of traffic sign is the output of the first branch, that is, the channel type corresponding to the probability point with the highest probability value in the first result;
交通标志的数值是第二分支的输出，即第二结果中，大于设定阈值的概率值中，具有最高概率值的概率点对应的通道数值；The value of the traffic sign is the output of the second branch, that is, in the second result, among the probability values greater than the set threshold, the channel value corresponding to the probability point with the highest probability value;
值得说明的是，对于部分交通标志类型而言，无需第二分支的输出即可确定该交通标志，即部分交通标志类型并不存在数字值和/或预设的定义数值。It should be noted that, for some types of traffic signs, the traffic signs can be determined without the output of the second branch, that is, some types of traffic signs do not have numerical values and/or preset defined values.
这种情况下，识别网络解码器层的第二分支输出概率应当小于设定的阈值，为了达到这一目的，可以在识别网络的训练过程中，通过损失函数的设置优化网络识别能力，从而使得识别网络的误检/错检率达到目标。In this case, the output probability of the second branch of the decoder layer of the recognition network should be less than the set threshold. In order to achieve this purpose, the network recognition ability can be optimized by setting the loss function during the training process of the recognition network, so that the Identify the false detection/false detection rate of the network to achieve the target.
更进一步地，对于训练后的识别网络，若出现上述情况，即第一分支的识别结果显示交通标志为不存在数字值和/或预设的定义数值的交通标志，而第二分支的识别结果显示交通标志为存在数字值和/或预设的定义数值的交通标志，则可以以第一分支和第二分支的概率值进行判断，采信概率值更高的识别结果，即：Further, for the trained recognition network, if the above situation occurs, that is, the recognition result of the first branch shows that the traffic sign is a traffic sign without a numerical value and/or a preset defined value, while the recognition result of the second branch If the displayed traffic sign is a traffic sign with a digital value and/or a preset defined value, the probability value of the first branch and the second branch can be used for judgment, and the recognition result with a higher probability value is adopted, that is:
若第一分支的识别结果概率值更高，则忽略第二分支的识别结果，基于第一结果得到交通标志识别结果；If the probability value of the recognition result of the first branch is higher, the recognition result of the second branch is ignored, and the traffic sign recognition result is obtained based on the first result;
若第二分支的识别结果概率值更高，或者第一分支和第二分支的识别结果概率值相等，则放弃本次交通标志识别结果，对输入图片进行调整(例如以下一帧图像为输入)重新进行识别。If the probability value of the recognition result of the second branch is higher, or the probability value of the recognition result of the first branch and the second branch is equal, then discard the current traffic sign recognition result and adjust the input picture (for example, the next frame of image is input) Re-identify.
本实施例的有益效果在于：The beneficial effects of this embodiment are:
通过设置第三分支、第四分支，能够基于同样的输入图片得出更为完整的结论，同时基于各分支的配合，能够对识别结果进行二次验证，从而得到更为准确的结论。By setting the third branch and the fourth branch, a more complete conclusion can be drawn based on the same input picture, and at the same time, based on the cooperation of each branch, the recognition result can be verified twice, so as to obtain a more accurate conclusion.
根据上述任一实施例，在本实施例中：According to any of the above embodiments, in this embodiment:
所述第一分支包括标志位置子分支、交通类型子分支；所述第一结果包括通过所述标志位置子分支得到的标志位置结果和通过所述交通类型子分支得到的交通类型结果。The first branch includes a sign position sub-branch and a traffic type sub-branch; the first result includes a sign position result obtained through the sign position sub-branch and a traffic type result obtained through the traffic type sub-branch.
所述标志位置子分支包括针对路面交通标志和非路面交通标志的2个通道；The sign position sub-branch includes 2 passages for road traffic signs and non-road traffic signs;
所述交通类型子分支包括针对最高限速类别、最低限速类别以及解除限速类别的3个通道。The traffic type sub-branch includes 3 channels for the highest speed limit category, the lowest speed limit category, and the release speed limit category.
本实施例进一步地对第一分支进行了细化，通过标志位置子分支、交通类型子分支的分离设置，进一步减少了神经网络所需的通道数量，提升了运算效率、减少了计算资源需求。This embodiment further refines the first branch, and further reduces the number of channels required by the neural network by separating the marking location sub-branch and the traffic type sub-branch, improving computing efficiency and reducing computing resource requirements.
值得说明的是，本实施例中的标志位置子分支、交通类型子分支虽然采用了“子分支”的命名方式进行说明，但在识别网络的结构设置中，这两个子分支与第二分支的地位相同、结构相似，子分支的命名并不能构成对本发明保护范围的限制。It is worth noting that although the sign location sub-branch and the traffic type sub-branch in this embodiment are described using the naming method of "sub-branch", in the structural setting of the identification network, the two sub-branches and the second branch are Having the same status and similar structure, the naming of the sub-branch does not constitute a limitation on the protection scope of the present invention.
进一步地，上述任一实施例中，所述识别网络均可以采用如图2所示的网络结构，即包括特征提取层和带有分支的解码器层的结构。Further, in any of the above embodiments, the identification network may adopt the network structure shown in FIG. 2 , that is, a structure including a feature extraction layer and a decoder layer with branches.
下面对本发明提供的基于类别映射的交通标志识别装置进行描述，下文描述的基于类别映射的交通标志识别装置与上文描述的基于类别映射的交通标志识别方法可相互对应参照。The traffic sign recognition device based on class mapping provided by the present invention will be described below. The traffic sign recognition device based on class map described below and the traffic sign recognition method based on class map described above may refer to each other correspondingly.
如图3所示，本发明实施例提供一种基于类别映射的交通标志识别系统，部署于车载终端，包括：As shown in FIG. 3 , an embodiment of the present invention provides a traffic sign recognition system based on class mapping, deployed in a vehicle terminal, including:
获取模块1，用于通过车载图像采集设备获取道路图像；The
特征提取模块2，用于将所述道路图像输入识别网络的特征提取层，得到提取结果；
解码模块3，用于将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支得到交通标志的第一结果、第二结果，并根据所述第一结果和第二结果得到交通标志识别结果；The
所述第一结果为所述交通标志的类别；所述第二结果为所述交通标志的数值；the first result is the category of the traffic sign; the second result is the value of the traffic sign;
所述识别网络是基于样本和标签训练得到的。The recognition network is trained based on samples and labels.
具体地，所述解码模块3包括：Specifically, the
四分单元，用于将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支、第三分支、第四分支得到交通标志的第一结果、第二结果、第三结果、第四结果，并根据所述第一结果、第二结果、第三结果以及第四结果得到交通标志识别结果；The quartet unit is used to input the extraction result into the decoder layer of the recognition network, and obtain the first result, the second result, the third result, and the fourth result, and obtain the traffic sign recognition result according to the first result, the second result, the third result and the fourth result;
所述第三结果为交通标志的宽高；所述第四结果为交通标志中心点坐标的偏移预测。The third result is the width and height of the traffic sign; the fourth result is the offset prediction of the coordinates of the center point of the traffic sign.
所述第一分支包括针对路面最高限速类别、路面最低限速类别、路面解除限速类别、非路面最高限速类别、非路面最低限速类别以及非路面解除限速类别的6个通道；The first branch includes 6 channels for the maximum speed limit category on the road, the minimum speed limit category on the road, the speed limit released on the road category, the maximum speed limit category off the road, the minimum speed limit category off the road, and the speed limit lifted off the road category;
所述第二分支包括针对交通数值的设定数量的通道；the second branch includes a set number of channels for traffic values;
所述第三分支包括针对交通标志宽、高的2个通道；The third branch includes 2 passages for the width and height of the traffic sign;
所述第四分支包括针对交通标志中心点在横、纵坐标轴方向偏移量的2个通道。The fourth branch includes two channels for the offset of the center point of the traffic sign in the direction of the horizontal and vertical axes.
所述交通数值是基于道路环境信息确定的。The traffic value is determined based on road environment information.
所述第一分支包括标志位置子分支、交通类型子分支；所述第一结果包括通过所述标志位置子分支得到的标志位置结果和通过所述交通类型子分支得到的交通类型结果。The first branch includes a sign position sub-branch and a traffic type sub-branch; the first result includes a sign position result obtained through the sign position sub-branch and a traffic type result obtained through the traffic type sub-branch.
所述标志位置子分支包括针对路面交通标志和非路面交通标志的2个通道；The sign position sub-branch includes 2 passages for road traffic signs and non-road traffic signs;
所述交通类型子分支包括针对最高限速类别、最低限速类别以及解除限速类别的3个通道。The traffic type sub-branch includes 3 channels for the highest speed limit category, the lowest speed limit category, and the release speed limit category.
图4示例了一种电子设备的实体结构示意图，如图4所示，该电子设备可以包括：处理器(processor)410、通信接口(Communications Interface)420、存储器(memory)430和通信总线440，其中，处理器410，通信接口420，存储器430通过通信总线440完成相互间的通信。处理器410可以调用存储器430中的逻辑指令，以执行基于类别映射的交通标志识别方法，该方法包括：通过车载图像采集设备获取道路图像；将所述道路图像输入识别网络的特征提取层，得到提取结果；将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支得到交通标志的第一结果、第二结果，并根据所述第一结果和第二结果得到交通标志识别结果；所述第一结果为所述交通标志的类别；所述第二结果为所述交通标志的数值；所述识别网络是基于样本和标签训练得到的。FIG. 4 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG. 4 , the electronic device may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430, and a
此外，上述的存储器430中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。In addition, the above-mentioned logic instructions in the
另一方面，本发明还提供一种计算机程序产品，所述计算机程序产品包括计算机程序，计算机程序可存储在非暂态计算机可读存储介质上，所述计算机程序被处理器执行时，计算机能够执行上述各方法所提供的基于类别映射的交通标志识别方法，该方法包括：通过车载图像采集设备获取道路图像；将所述道路图像输入识别网络的特征提取层，得到提取结果；将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支得到交通标志的第一结果、第二结果，并根据所述第一结果和第二结果得到交通标志识别结果；所述第一结果为所述交通标志的类别；所述第二结果为所述交通标志的数值；所述识别网络是基于样本和标签训练得到的。In another aspect, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can Execute the traffic sign recognition method based on category mapping provided by the above methods, the method includes: obtaining road images through a vehicle-mounted image acquisition device; inputting the road images into a feature extraction layer of a recognition network to obtain an extraction result; The result is input into the decoder layer of the recognition network, the first and second results of the traffic sign are obtained through the first branch and the second branch of the decoder layer, respectively, and the traffic sign is obtained according to the first and second results. Recognition result; the first result is the category of the traffic sign; the second result is the value of the traffic sign; the recognition network is obtained by training based on samples and labels.
又一方面，本发明还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现以执行上述各方法提供的基于类别映射的交通标志识别方法，该方法包括：通过车载图像采集设备获取道路图像；将所述道路图像输入识别网络的特征提取层，得到提取结果；将所述提取结果输入识别网络的解码器层，分别通过所述解码器层的第一分支、第二分支得到交通标志的第一结果、第二结果，并根据所述第一结果和第二结果得到交通标志识别结果；所述第一结果为所述交通标志的类别；所述第二结果为所述交通标志的数值；所述识别网络是基于样本和标签训练得到的。In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored, and the computer program is implemented by a processor to execute the class mapping-based traffic sign recognition method provided by the above methods. , the method includes: acquiring road images through a vehicle-mounted image acquisition device; inputting the road images into a feature extraction layer of a recognition network to obtain an extraction result; inputting the extraction results into a decoder layer of the recognition network, and passing the The first branch and the second branch of the layer obtain the first result and the second result of the traffic sign, and obtain the traffic sign recognition result according to the first result and the second result; the first result is the type of the traffic sign ; the second result is the value of the traffic sign; the recognition network is obtained by training based on samples and labels.
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic A disc, an optical disc, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or some parts of the embodiments.
最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111467675.3ACN114419576A (en) | 2021-12-03 | 2021-12-03 | Traffic sign identification method and system based on category mapping |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111467675.3ACN114419576A (en) | 2021-12-03 | 2021-12-03 | Traffic sign identification method and system based on category mapping |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114419576Atrue CN114419576A (en) | 2022-04-29 |
Family
ID=81264711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111467675.3APendingCN114419576A (en) | 2021-12-03 | 2021-12-03 | Traffic sign identification method and system based on category mapping |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114419576A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609485A (en)* | 2017-08-16 | 2018-01-19 | 中国科学院自动化研究所 | The recognition methods of traffic sign, storage medium, processing equipment |
| US20190026607A1 (en)* | 2016-03-29 | 2019-01-24 | Alibaba Group Holding Limited | Character recognition method and device |
| CN109840557A (en)* | 2019-01-24 | 2019-06-04 | 深圳爱莫科技有限公司 | Image-recognizing method and device |
| CN111401466A (en)* | 2020-03-26 | 2020-07-10 | 广州紫为云科技有限公司 | Traffic sign detection and identification marking method and device and computer equipment |
| CN111400522A (en)* | 2020-04-29 | 2020-07-10 | 广州紫为云科技有限公司 | Traffic sign recognition method, training method and equipment |
| CN112580628A (en)* | 2020-12-22 | 2021-03-30 | 浙江智慧视频安防创新中心有限公司 | License plate character recognition method and system based on attention mechanism |
| CN113159039A (en)* | 2021-02-09 | 2021-07-23 | 北京市商汤科技开发有限公司 | Image recognition method and device, electronic equipment and storage medium |
| CN113591543A (en)* | 2021-06-08 | 2021-11-02 | 广西综合交通大数据研究院 | Traffic sign recognition method and device, electronic equipment and computer storage medium |
- 2021
- 2021-12-03CNCN202111467675.3Apatent/CN114419576A/enactivePending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190026607A1 (en)* | 2016-03-29 | 2019-01-24 | Alibaba Group Holding Limited | Character recognition method and device |
| CN107609485A (en)* | 2017-08-16 | 2018-01-19 | 中国科学院自动化研究所 | The recognition methods of traffic sign, storage medium, processing equipment |
| CN109840557A (en)* | 2019-01-24 | 2019-06-04 | 深圳爱莫科技有限公司 | Image-recognizing method and device |
| CN111401466A (en)* | 2020-03-26 | 2020-07-10 | 广州紫为云科技有限公司 | Traffic sign detection and identification marking method and device and computer equipment |
| CN111400522A (en)* | 2020-04-29 | 2020-07-10 | 广州紫为云科技有限公司 | Traffic sign recognition method, training method and equipment |
| CN112580628A (en)* | 2020-12-22 | 2021-03-30 | 浙江智慧视频安防创新中心有限公司 | License plate character recognition method and system based on attention mechanism |
| CN113159039A (en)* | 2021-02-09 | 2021-07-23 | 北京市商汤科技开发有限公司 | Image recognition method and device, electronic equipment and storage medium |
| CN113591543A (en)* | 2021-06-08 | 2021-11-02 | 广西综合交通大数据研究院 | Traffic sign recognition method and device, electronic equipment and computer storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 李昕蔚;丁正彦;尚岩峰;祝永新;汪辉;钟雪霞;田犁;黄尊恺;封松林;: "一种针对路口监控图像的区域分割方法", 计算机应用与软件, no. 03, 12 March 2020 (2020-03-12), pages 236 - 243* |
| 谢锦;蔡自兴;邓海涛;盛艳;: "基于图像不变特征深度学习的交通标志分类", 计算机辅助设计与图形学学报, no. 04, 15 April 2017 (2017-04-15), pages 632 - 640* |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111815959B (en) | Vehicle violation detection method and device and computer readable storage medium | |
| CN105930791B (en) | Recognition method of road traffic signs based on multi-camera fusion based on DS evidence theory | |
| CN103324930B (en) | A license plate character segmentation method based on gray histogram binarization | |
| CN110956081B (en) | Method and device for identifying position relationship between vehicle and traffic marking and storage medium | |
| CN114463715B (en) | A lane line detection method | |
| CN114120289B (en) | Method and system for identifying driving area and lane line | |
| CN112507852A (en) | Lane line identification method, device, equipment and storage medium | |
| CN111435446A (en) | License plate identification method and device based on L eNet | |
| CN112613434B (en) | Road target detection method, device and storage medium | |
| CN109711372A (en) | A kind of recognition methods of lane line and system, storage medium, server | |
| CN111553368A (en) | Fake license plate recognition method, fake license plate training method, fake license plate recognition device, fake license plate recognition equipment and storage medium | |
| CN114419576A (en) | Traffic sign identification method and system based on category mapping | |
| CN118781450A (en) | A method, device, equipment and medium for constructing question and answer data based on a large model | |
| CN115205810A (en) | A road traffic sign recognition method and system, and a computer-readable storage medium | |
| CN112509321A (en) | Unmanned aerial vehicle-based driving control method and system for urban complex traffic situation and readable storage medium | |
| CN117612117A (en) | Roadside near weed segmentation method, system and medium based on vehicle-mounted recorder | |
| CN114495056B (en) | Parking lot post detection method, detection device, vehicle and storage medium | |
| CN116665186A (en) | Vehicle automatic driving scene data labeling method, system, server and medium | |
| CN114119951B (en) | Vehicle information labeling method, device, equipment and storage medium | |
| CN112434591B (en) | Lane line determination method and device | |
| CN115620047A (en) | Target object attribute information determination method and device, electronic equipment and storage medium | |
| CN111368784B (en) | Target identification method, device, computer equipment and storage medium | |
| CN113591670A (en) | Lane line detection method based on convolutional neural network | |
| CN114399740A (en) | Universal traffic sign identification method and system based on dynamic network structure | |
| CN117710913A (en) | Two-dimensional lane line identification method and device, storage medium, equipment and vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |