技术领域Technical field
本公开涉及人工智能技术领域，具体为计算机视觉、虚拟现实、深度学习、大模型等技术领域，可应用于自动驾驶等场景，尤其涉及一种基于点云的目标检测方法、装置、设备、介质及产品。The present disclosure relates to the field of artificial intelligence technology, specifically to the technical fields of computer vision, virtual reality, deep learning, large models, etc., which can be applied to scenarios such as autonomous driving, and in particular, to a point cloud-based target detection method, device, equipment, and medium and products.
背景技术Background technique
3D点云(3-dimensional point cloud)是用于在同一空间参考下表达空间分布的点云数据集合。3D点云的应用主要是基于3D点云实现的目标检测，可以将检测结果应用于自动驾驶、游戏定位、机器人感知、三维重建等场景。3D point cloud (3-dimensional point cloud) is a collection of point cloud data used to express spatial distribution under the same spatial reference. The application of 3D point cloud is mainly target detection based on 3D point cloud. The detection results can be applied to scenarios such as autonomous driving, game positioning, robot perception, and three-dimensional reconstruction.
目前，可以从3D点云中检测包围对象的目标框，例如检测3D点云中包含车辆的目标框，该目标框可以为长方体。但是，从3D点云中检测获得的目标框不够准确，精度不高。Currently, a target frame surrounding an object can be detected from a 3D point cloud. For example, a target frame containing a vehicle in a 3D point cloud can be detected. The target frame can be a cuboid. However, the target frame detected from the 3D point cloud is not accurate enough and the accuracy is not high.
发明内容Contents of the invention
本公开提供了一种基于点云的目标检测方法、装置、设备、介质及产品。The present disclosure provides a point cloud-based target detection method, device, equipment, media and products.
根据本公开的第一方面，提供了一种基于点云的目标检测方法，包括：According to a first aspect of the present disclosure, a point cloud-based target detection method is provided, including:
获取从3D点云中获得的检测框，所述检测框为包围所述3D点云中目标对象的检测框；Obtain a detection frame obtained from the 3D point cloud, where the detection frame is a detection frame surrounding the target object in the 3D point cloud;
基于训练集中的多个标注框，筛选与所述检测框相匹配的至少一个目标标注框；Based on the multiple annotation boxes in the training set, filter at least one target annotation box that matches the detection frame;
根据至少一个所述目标标注框对所述检测框进行校验处理，获得目标检测框。The detection frame is verified according to at least one of the target annotation frames to obtain a target detection frame.
根据本公开的第二方面，提供了一种基于点云的目标检测装置，包括：According to a second aspect of the present disclosure, a point cloud-based target detection device is provided, including:
获取单元，用于获取从3D点云中获得的检测框，所述检测框为包围所述3D点云中目标对象的检测框；An acquisition unit, used to acquire a detection frame obtained from the 3D point cloud, where the detection frame is a detection frame surrounding the target object in the 3D point cloud;
筛选单元，用于基于训练集中的多个标注框，筛选与所述检测框相匹配的至少一个目标标注框；A screening unit configured to filter at least one target annotation frame that matches the detection frame based on the multiple annotation frames in the training set;
校正单元，用于根据至少一个所述目标标注框对所述检测框进行校验处理，获得目标检测框。A correction unit, configured to perform verification processing on the detection frame according to at least one of the target annotation frames to obtain a target detection frame.
根据本公开的第三方面，提供了一种电子设备，包括：According to a third aspect of the present disclosure, an electronic device is provided, including:
至少一个处理器；以及at least one processor; and
与所述至少一个处理器通信连接的存储器；其中，a memory communicatively connected to the at least one processor; wherein,
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行第一方面以及第一方面各种可能所述的方法。The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can perform the first aspect and various possibilities of the first aspect. method described.
根据本公开的第四方面，一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行第一方面以及第一方面各种可能所述的方法。According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the first aspect and various possible methods described in the first aspect. .
根据本公开的第五方面，提供了一种计算机程序产品，所述计算机程序产品包括：计算机程序，所述计算机程序存储在可读存储介质中，电子设备的至少一个处理器可以从所述可读存储介质读取所述计算机程序，所述至少一个处理器执行所述计算机程序使得电子设备执行第一方面所述的方法。According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can obtain Reading the storage medium reads the computer program, and the at least one processor executes the computer program to cause the electronic device to perform the method described in the first aspect.
根据本公开的技术方案，可以获取从3D点云中获得的检测框。该检测框可以为包围3D点云中目标对象的检测框，实现目标对象的初步检测。之后，可以基于训练集中的多个标注框，筛选与检测框相匹配的至少一个目标标注框，进而根据至少一个目标标注框对检测框进行校验处理，获得目标检测框。至少一个目标标注框来源于训练集中的多个标注框，可以体现训练集中的真实检测框，可以准确表示对象所在区域，进而利用至少一个目标标注框对检测框进行校验处理，可以使得目标检测框与目标对象的真实框更接近，可以有效提高目标检测框的预测精度和准确度。According to the technical solution of the present disclosure, the detection frame obtained from the 3D point cloud can be obtained. The detection frame can be a detection frame surrounding the target object in the 3D point cloud to achieve preliminary detection of the target object. Afterwards, based on multiple annotation frames in the training set, at least one target annotation frame that matches the detection frame can be screened, and then the detection frame can be verified based on at least one target annotation frame to obtain the target detection frame. At least one target annotation frame is derived from multiple annotation frames in the training set, which can reflect the real detection frame in the training set, and can accurately represent the area where the object is located, and then use at least one target annotation frame to verify the detection frame, which can enable target detection The frame is closer to the real frame of the target object, which can effectively improve the prediction accuracy and accuracy of the target detection frame.
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.
附图说明Description of drawings
附图用于更好地理解本方案，不构成对本公开的限定。其中：The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. in:
图1是根据本公开第一实施例的示意图；Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;
图2是根据本公开第二实施例的示意图；Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;
图3a示出了一种检测框的示意图；Figure 3a shows a schematic diagram of a detection frame;
图3b示出了一种标注框的示意图；Figure 3b shows a schematic diagram of a labeling box;
图4是根据本公开第三实施例的示意图；Figure 4 is a schematic diagram according to a third embodiment of the present disclosure;
图5是根据本公开第四实施例的示意图；Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;
图6是根据本公开实施例提供的一种基于点云的目标检测方法的应用场景图；Figure 6 is an application scenario diagram of a point cloud-based target detection method provided according to an embodiment of the present disclosure;
图7是根据本公开第五实施例的示意图；Figure 7 is a schematic diagram according to a fifth embodiment of the present disclosure;
图8是用来实现本公开实施例的基于点云的目标检测方法的电子设备的框图。FIG. 8 is a block diagram of an electronic device used to implement a point cloud-based target detection method according to an embodiment of the present disclosure.
具体实施方式Detailed ways
以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本公开的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered to be exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.
本公开提供一种基于点云的目标检测方法、装置、设备、介质及产品，可以应用于人工智能技术领域，具体为计算机视觉、虚拟现实、深度学习、大模型等技术领域，可应用于自动驾驶等场景，以达到精准识别目标对象的检测框的目标。The present disclosure provides a point cloud-based target detection method, device, equipment, medium and product, which can be applied in the field of artificial intelligence technology, specifically computer vision, virtual reality, deep learning, large models and other technical fields, and can be used in automatic Driving and other scenes to achieve the goal of accurately identifying the detection frame of the target object.
相关技术中，3D点云(3dimensional point cloud)是用于在同一空间参考下表达空间分布的点云数据集合。3D点云数据中可以提供三维空间的数据。通过3D点云可以感知汽车周边的道路环境，进行目标检测，实现目标追踪等现象。目前，点云数据的主要应用场景是，从3D点云中识别包围目标对象的检测框，通过检测框进行目标对象的追踪、轨迹识别和规划、避障等处理。较为常用的目标对象的检测算法一般是通过深度学习模型从3D点云中识别包围目标对象的检测框，这种检测方式较为粗糙，获得的检测框与目标对象所在实际区域存在尺寸、方向上的偏差，检测精度不高。In related art, 3D point cloud (3dimensional point cloud) is a point cloud data set used to express spatial distribution under the same spatial reference. 3D point cloud data can provide data in three-dimensional space. Through 3D point cloud, the road environment around the car can be sensed, target detection can be performed, and target tracking can be achieved. At present, the main application scenario of point cloud data is to identify the detection frame surrounding the target object from the 3D point cloud, and use the detection frame to perform target object tracking, trajectory identification and planning, obstacle avoidance and other processing. The more commonly used target object detection algorithm generally uses a deep learning model to identify the detection frame surrounding the target object from the 3D point cloud. This detection method is relatively rough, and there are differences in size and direction between the obtained detection frame and the actual area where the target object is located. Deviation, detection accuracy is not high.
为了解决上述问题，本公开的技术方案中，在获得检测框之后，考虑对检测框进行优化，获得更准确的检测框。为了解决检测框优化问题，本公开的技术方案考虑到，训练集中存在多个标注框，可以利用多个标注框中与检测框相匹配的目标标注框，对检测框进行校验处理。使得检测框的校正过程参考属于真实框的至少一个目标标注框，获得的目标检测框更贴近于真实框，实现检测框的精确校正。In order to solve the above problem, in the technical solution of the present disclosure, after obtaining the detection frame, the detection frame is considered to be optimized to obtain a more accurate detection frame. In order to solve the detection frame optimization problem, the technical solution of the present disclosure takes into account that there are multiple annotation frames in the training set, and the detection frame can be verified by using the target annotation frame in the multiple annotation frames that matches the detection frame. The correction process of the detection frame is made to refer to at least one target annotation frame belonging to the real frame, and the obtained target detection frame is closer to the real frame, thereby achieving accurate correction of the detection frame.
本公开的技术方案，可以获取从3D点云中获得的检测框。该检测框可以为包围3D点云中目标对象的检测框，实现目标对象的初步检测。之后，可以基于训练集中的多个标注框，筛选与检测框相匹配的至少一个目标标注框，进而根据至少一个目标标注框对检测框进行校验处理，获得目标检测框。至少一个目标标注框来源于训练集中的多个标注框，可以体现训练集中的真实检测框，可以准确表示对象所在区域，进而利用至少一个目标标注框对检测框进行校验处理，可以使得目标检测框与目标对象的真实框更接近，可以有效提高目标检测框的预测精度和准确度。The technical solution of the present disclosure can obtain detection frames obtained from 3D point clouds. The detection frame can be a detection frame surrounding the target object in the 3D point cloud to achieve preliminary detection of the target object. Afterwards, based on multiple annotation frames in the training set, at least one target annotation frame that matches the detection frame can be screened, and then the detection frame can be verified based on at least one target annotation frame to obtain the target detection frame. At least one target annotation frame is derived from multiple annotation frames in the training set, which can reflect the real detection frame in the training set, and can accurately represent the area where the object is located, and then use at least one target annotation frame to verify the detection frame, which can enable target detection The frame is closer to the real frame of the target object, which can effectively improve the prediction accuracy and accuracy of the target detection frame.
下面将结合附图对本申请的技术方案进行详细说明。The technical solution of the present application will be described in detail below with reference to the accompanying drawings.
图1为本公开第一实施例的示意图，参考图1所示的用于实现基于点云的目标检测方法的系统示意图。该系统可以包括：电子设备1和车辆2。车辆2的车顶上可以配置有3D扫描设备3，3D扫描设备例如可以为激光雷达、RGB(Red Green Blue)双目相机、3D结构光相机、tof(time-of-flight camera)相机等设备。车辆2行驶过程中，3D扫描设备3可以采集车辆2前方的3D点云，并将3D点云传输至电子设备1。FIG. 1 is a schematic diagram of a first embodiment of the present disclosure. Refer to the system schematic diagram shown in FIG. 1 for implementing a point cloud-based target detection method. The system may include: an electronic device 1 and a vehicle 2 . A 3D scanning device 3 may be configured on the roof of the vehicle 2. The 3D scanning device may be, for example, a laser radar, an RGB (Red Green Blue) binocular camera, a 3D structured light camera, a tof (time-of-flight camera) camera, or other equipment. . While the vehicle 2 is driving, the 3D scanning device 3 can collect the 3D point cloud in front of the vehicle 2 and transmit the 3D point cloud to the electronic device 1 .
电子设备1可以接收3D点云，并从3D点云中获得的检测框。检测框可以为包围3D点云中目标对象的检测框。之后，基于训练集中的多个标注框，筛选与检测框相匹配的至少一个目标标注框，至少一个目标标注框可以为训练数据的真实框。进而利用至少一个目标标注框对检测框进行校验处理，获得目标检测框，可以使得检测框更接近于目标对象的真实框，有效提高目标检测框的预测精度和准确度。The electronic device 1 can receive the 3D point cloud and obtain the detection frame from the 3D point cloud. The detection frame may be a detection frame surrounding the target object in the 3D point cloud. Afterwards, based on the multiple label boxes in the training set, at least one target label box that matches the detection frame is screened. At least one target label box can be a true frame of the training data. Then, at least one target annotation frame is used to perform verification processing on the detection frame to obtain the target detection frame, which can make the detection frame closer to the real frame of the target object and effectively improve the prediction accuracy and accuracy of the target detection frame.
图2为本公开第二实施例的示意图，参考图2所示的基于点云的目标检测方法，可以配置为基于点云的目标检测装置，该装置可以位于电子设备中。基于点云的目标检测方法可以包括下列步骤：Figure 2 is a schematic diagram of a second embodiment of the present disclosure. Referring to the point cloud-based target detection method shown in Figure 2, it can be configured as a point cloud-based target detection device, and the device can be located in an electronic device. The point cloud-based target detection method can include the following steps:
201、获取从3D点云中获得的检测框，检测框为包围3D点云中目标对象的检测框。201. Obtain the detection frame obtained from the 3D point cloud. The detection frame is the detection frame surrounding the target object in the 3D point cloud.
本实施例中，3D点云可以是指通过三维空间表示的数据，可以由三维空间中的多个点云数据组成。3D点云可以通过激光雷达扫或者其它3D扫描设备产生，可以应用于自动驾驶、增强现实、计算机视觉等技术领域。In this embodiment, the 3D point cloud may refer to data represented by a three-dimensional space, and may be composed of multiple point cloud data in the three-dimensional space. 3D point clouds can be generated through lidar scanning or other 3D scanning equipment, and can be used in technical fields such as autonomous driving, augmented reality, and computer vision.
其中，步骤201之前，还包括：接收3D扫描设备发送的3D点云。3D扫描设备例如可以包括激光雷达或其它3D扫描设备。Among them, before step 201, it also includes: receiving the 3D point cloud sent by the 3D scanning device. The 3D scanning device may include, for example, a lidar or other 3D scanning device.
可选地，获取从3D点云中获得的检测框，包括：通过目标检测算法，检测3D点云中的检测框，以获得3D点云中的检测框。目标检测算法可以为神经网络算法、机器学习算法中的任一种用于3d点云检测的算法，例如可以为PointPillars(尖柱)算法、Deep3Dbox(深度三维盒)检测算法等任一种，本实施例中对目标检测算法的具体类型并不过多限定。Optionally, obtaining the detection frame obtained from the 3D point cloud includes: detecting the detection frame in the 3D point cloud through a target detection algorithm to obtain the detection frame in the 3D point cloud. The target detection algorithm can be any of the neural network algorithm and the machine learning algorithm used for 3D point cloud detection, for example, it can be any of the PointPillars (pointed pillar) algorithm, Deep3Dbox (depth three-dimensional box) detection algorithm, etc. The specific type of target detection algorithm is not too limited in the embodiment.
本实施例中，检测框可以为三维的长方体框，标注框与检测框可以具有相同的空间维度，标注框可以为三维的长方体框。In this embodiment, the detection frame may be a three-dimensional cuboid frame, the labeling frame and the detection frame may have the same spatial dimension, and the labeling frame may be a three-dimensional cuboid frame.
202、基于训练集中的多个标注框，筛选与检测框相匹配的至少一个目标标注框。202. Based on the multiple annotation frames in the training set, filter at least one target annotation frame that matches the detection frame.
可选地，检测框可以关联目标对象的类别。以自动驾驶领域为例，目标对象的类别可以包括车辆、路灯、行人、建筑物等任一种。Optionally, the detection box can be associated with the category of the target object. Taking the field of autonomous driving as an example, target object categories can include vehicles, street lights, pedestrians, buildings, etc.
训练集可以包括多个训练数据，每个训练数据可以包括至少一个标注框，标注框可以是指包围训练数据中的对象的真实框，可以通过人工标注或者自动标注获得。每个标注框可以分别关联对象类别。The training set may include multiple training data, and each training data may include at least one annotation box. The annotation box may refer to a real box surrounding an object in the training data, and may be obtained through manual annotation or automatic annotation. Each label box can be associated with an object category respectively.
为了使得步骤202中的多个标注框参与到目标对象的预测时，获得更有效的优化效果，步骤202之前，可以从训练集中获取对象类别与目标对象的类别相同的多个标注框。例如，目标对象为车辆，多个标注框分别标注的对象类别可以均为车辆。In order to obtain a more effective optimization effect when multiple annotation boxes in step 202 participate in the prediction of the target object, before step 202, multiple annotation boxes whose object category is the same as the target object category can be obtained from the training set. For example, if the target object is a vehicle, the object categories marked by multiple labeling boxes can all be vehicles.
可选地，步骤202，可以包括：基于训练集中的多个标注框，筛选满足检测框对应的匹配条件的至少一个目标标注框。Optionally, step 202 may include: screening at least one target annotation box that satisfies the matching condition corresponding to the detection frame based on the multiple annotation boxes in the training set.
203、根据至少一个目标标注框对检测框进行校验处理，获得目标检测框。203. Verify the detection frame based on at least one target annotation frame to obtain the target detection frame.
可选地，步骤203，可以包括：利用检测框，对至少一个目标标注框分别进行校正处理，获得至少一个校正框。从至少一个校正框和检测框中确定满足校验条件的目标检测框。校验条件可以是指检测框或者校正框包含的点云数量最大。点云数量可以是指检测框或校正框在3D点云中所包围的点的数量。Optionally, step 203 may include: using the detection frame to perform correction processing on at least one target annotation frame, respectively, to obtain at least one correction frame. Determine a target detection frame that satisfies the verification condition from at least one correction frame and detection frame. The verification condition may refer to the maximum number of point clouds contained in the detection frame or correction frame. The number of point clouds may refer to the number of points surrounded by the detection frame or correction frame in the 3D point cloud.
目标检测框可以为对检测框进行校验处理获得的长方体。The target detection frame may be a cuboid obtained by performing verification processing on the detection frame.
本实施例中，在步骤203之后，上述基于点云的目标检测方法还可以包括：根据目标检测框，对目标对象进行位置追踪处理，获得目标对象的运动轨迹，并输出目标对象的运动轨迹。In this embodiment, after step 203, the above point cloud-based target detection method may also include: performing position tracking processing on the target object according to the target detection frame, obtaining the movement trajectory of the target object, and outputting the movement trajectory of the target object.
该方法还可以包括：根据目标检测框，对目标对象进行导航处理，获得目标对象的行驶路径，并根据行驶路径，生成导航电子地图，输出导航电子地图，以便于车辆驾驶。The method may also include: performing navigation processing on the target object according to the target detection frame, obtaining the driving path of the target object, generating a navigation electronic map according to the driving path, and outputting the navigation electronic map to facilitate vehicle driving.
此外，获得目标对象的行驶路径之后，可以根据目标对象的行驶路径，控制目标对象在该行驶路径自动驾驶。In addition, after obtaining the driving path of the target object, the target object can be controlled to drive automatically on the driving path according to the driving path of the target object.
通过上述应用场景，对目标检测框在自动驾驶、交通领域的应用简单举例，可以实现目标检测框的广泛应用。Through the above application scenarios, a simple example of the application of target detection frames in the fields of autonomous driving and transportation can be used to realize the wide application of target detection frames.
本公开的技术方案，可以获取从3D点云中检测获得的检测框。该检测框可以为包围3D点云中目标对象的检测框，实现目标对象的初步检测。之后，可以基于训练集中的多个标注框，筛选与检测框相匹配的至少一个目标标注框，进而根据至少一个目标标注框对检测框进行校验处理，获得目标检测框。至少一个目标标注框来源于训练集中的多个标注框，可以体现训练集中的真实检测框，可以准确表示对象所在区域，进而利用至少一个目标标注框对检测框进行校验处理，可以使得目标检测框与目标对象的真实框更接近，可以有效提高目标检测框的预测精度和准确度。The technical solution of the present disclosure can obtain detection frames detected from 3D point clouds. The detection frame can be a detection frame surrounding the target object in the 3D point cloud to achieve preliminary detection of the target object. Afterwards, based on multiple annotation frames in the training set, at least one target annotation frame that matches the detection frame can be screened, and then the detection frame can be verified based on at least one target annotation frame to obtain the target detection frame. At least one target annotation frame is derived from multiple annotation frames in the training set, which can reflect the real detection frame in the training set, and can accurately represent the area where the object is located, and then use at least one target annotation frame to verify the detection frame, which can enable target detection The frame is closer to the real frame of the target object, which can effectively improve the prediction accuracy and accuracy of the target detection frame.
为使读者更深刻地理解本公开的实现原理，现结合以下图3-图6对图2所示的实施例进行进一步细化。In order to allow readers to have a deeper understanding of the implementation principle of the present disclosure, the embodiment shown in Figure 2 is further detailed with reference to the following Figures 3-6.
作为一个实施例，根据至少一个目标标注框对检测框进行校验处理，获得目标检测框，可以包括：As an embodiment, performing verification processing on the detection frame based on at least one target annotation frame to obtain the target detection frame may include:
将至少一个目标标注框分别变换为与检测框的几何信息相匹配的校正框，获得至少一个校正框；Transform at least one target annotation frame into a correction frame that matches the geometric information of the detection frame, and obtain at least one correction frame;
基于至少一个校正框和检测框，确定目标检测框。Based on at least one correction frame and a detection frame, a target detection frame is determined.
本实施例中，每个框具有几何信息，任意框的几何信息是指该框的几何参数构成的信息。几何参数例如可以包括以下参数中的至少一个：In this embodiment, each frame has geometric information, and the geometric information of any frame refers to the information composed of the geometric parameters of the frame. Geometric parameters may include, for example, at least one of the following parameters:
尺寸(长、宽、高)、形状、方向、中心点坐标等参数。方向可以使用航向角等角度表示，航向角可以是指检测框在地平面的偏航角。Parameters such as size (length, width, height), shape, direction, center point coordinates, etc. The direction can be expressed by angles such as heading angle, which can refer to the yaw angle of the detection frame on the ground plane.
从多个标注框中初步筛选获得的至少一个标注框，该至少一个标注框各自的几何信息与检测框的几何信息可能不同。直接利用标注框锚定目标对象，实际并不能准确圈定目标对象所在区域。因此，可以利用检测框的几何信息对目标标注框进行几何变换，以获得校正框，使得校正框的几何信息与检测框的几何信息保持一致。At least one annotation frame is obtained by preliminary screening from multiple annotation frames. The geometric information of each annotation frame of the at least one annotation frame may be different from the geometric information of the detection frame. Directly using the annotation box to anchor the target object cannot actually accurately delineate the area where the target object is located. Therefore, the geometric information of the detection frame can be used to perform geometric transformation on the target annotation frame to obtain the correction frame, so that the geometric information of the correction frame is consistent with the geometric information of the detection frame.
示例性地，图3a示出了一个检测框301和图3b示出了一个标注框302，图3a所示的检测框301与图3b所示的标注框302的方向和尺寸不同。For example, FIG. 3a shows a detection frame 301 and FIG. 3b shows a labeling frame 302. The detection frame 301 shown in FIG. 3a and the labeling frame 302 shown in FIG. 3b have different directions and sizes.
本实施例中，目标标注框的变换步骤可以包括：计算目标标注框的几何信息和检测框的几何信息之间的映射关系，根据该映射关系将目标标注框变换为与检测框的几何信息相匹配的校正框。In this embodiment, the step of transforming the target annotation frame may include: calculating a mapping relationship between the geometric information of the target annotation frame and the geometric information of the detection frame, and transforming the target annotation frame into a state corresponding to the geometric information of the detection frame according to the mapping relationship. Matching correction frame.
可选地，基于至少一个校正框和检测框，确定目标检测框，可以包括：从至少一个校正框和检测框中选择目标检测框。Optionally, determining the target detection frame based on at least one correction frame and detection frame may include: selecting the target detection frame from at least one correction frame and detection frame.
进一步地，从至少一个校正框和检测框中选择目标检测框，可以包括：将至少一个校正框和检测框作为多个候选框，对多个候选框分别进行检测质量评价，获得多个候选框分别对应的质量检测结果，获得质量检测结果最高的框为目标检测框。Further, selecting the target detection frame from at least one correction frame and detection frame may include: using at least one correction frame and detection frame as multiple candidate frames, performing detection quality evaluation on the multiple candidate frames respectively, and obtaining multiple candidate frames. Corresponding quality detection results respectively, the frame with the highest quality detection result is the target detection frame.
其中，对多个候选框分别进行检测质量评价，获得多个候选框分别对应的质量检测结果，可以包括：对多个候选框分别对目标对象的包围效果进行质量评价，获得多个候选框分别对应的质量评价结果。Among them, performing a detection quality evaluation on multiple candidate frames respectively, and obtaining quality detection results corresponding to the multiple candidate frames, may include: performing a quality evaluation on the surrounding effect of the target object on the multiple candidate frames, and obtaining the quality detection results of the multiple candidate frames respectively. Corresponding quality evaluation results.
本公开的技术方案，将目标标注框变换为校正框时，使用了检测框的几何信息，使得校正框与检测框的几何信息相匹配，进而基于至少一个矫正框和检测框确定的目标检测框时，增加了目标检测框的选择空间，相比于直接将检测框作为预测结果，可以提供更多的可选框，选择更准确的框作为目标检测框，有效提高目标检测框的精度。The technical solution of the present disclosure uses the geometric information of the detection frame when converting the target annotation frame into the correction frame, so that the correction frame matches the geometric information of the detection frame, and then determines the target detection frame based on at least one correction frame and the detection frame. At the same time, the selection space of the target detection frame is increased. Compared with directly using the detection frame as the prediction result, more optional frames can be provided, and a more accurate frame can be selected as the target detection frame, effectively improving the accuracy of the target detection frame.
图4为本公开第三实施例的示意图，与前述实施例的不同之处在于，基于至少一个校正框和检测框，确定目标检测框，包括：Figure 4 is a schematic diagram of a third embodiment of the present disclosure. The difference from the previous embodiment is that the target detection frame is determined based on at least one correction frame and a detection frame, including:
401、基于至少一个校正框和检测框，确定至少一个候选框。401. Determine at least one candidate frame based on at least one correction frame and detection frame.
在一种可能的设计中，步骤401，可以包括：确定至少一个校正框和检测框为至少一个候选框。In a possible design, step 401 may include: determining at least one correction frame and detection frame as at least one candidate frame.
在又一种可能的设计中，步骤401还可以包括：将至少一个校正框和检测框，结合3D点云，生成选择页面，选择页面可以包括3D点云以及位于在该3D点云上显示的至少一个校正框和检测框。响应于用户针对至少一个校正框和检测框执行的框选择操作，获得用户选择的至少一个候选框。In another possible design, step 401 may also include: combining at least one correction frame and a detection frame with a 3D point cloud to generate a selection page. The selection page may include a 3D point cloud and a location displayed on the 3D point cloud. At least one calibration frame and detection frame. In response to a frame selection operation performed by the user on at least one correction frame and detection frame, at least one candidate frame selected by the user is obtained.
在又一种可能的设计中，步骤401还可以包括：从至少一个校正框和检测框中确定满足使用条件的至少一个候选框。使用条件可以包括校正框或检测框在3D点云所对应的包围范围内的点云数量大于或等于数量阈值。数量阈值可以根据使用条件设置，数量阈值越大，校正框或检测框在3D点云所对应包围范围内的点更多，聚合形成的目标对象的形状更准确。In yet another possible design, step 401 may further include: determining at least one candidate frame that satisfies the usage condition from at least one correction frame and detection frame. The conditions for use may include that the number of point clouds in the correction frame or detection frame within the surrounding range corresponding to the 3D point cloud is greater than or equal to the quantity threshold. The quantity threshold can be set according to the usage conditions. The larger the quantity threshold, the more points the correction frame or detection frame will have within the corresponding surrounding range of the 3D point cloud, and the shape of the target object formed by aggregation will be more accurate.
402、确定至少一个候选框分别在3D点云对应的点云数量，点云数量为候选框在3D点云包含的点云数据的数量。402. Determine the number of point clouds corresponding to at least one candidate frame in the 3D point cloud. The number of point clouds is the number of point cloud data contained in the 3D point cloud of the candidate frame.
本实施例中，步骤402中任意候选框的点云数量的确定步骤，可以包括：获取3D点云中的至少一个点云数据，根据至少一个点云数据分别对应的坐标，确定坐标位于候选框的坐标范围内的目标点云数据，计算目标点云数据的点云数量，获得该候选框在3D点云对应的点云数量。In this embodiment, the step of determining the number of point clouds in any candidate frame in step 402 may include: obtaining at least one point cloud data in the 3D point cloud, and determining that the coordinates are located in the candidate frame based on the coordinates corresponding to the at least one point cloud data. Target point cloud data within the coordinate range, calculate the number of point clouds of the target point cloud data, and obtain the number of point clouds corresponding to the candidate frame in the 3D point cloud.
点云数据可以是指3D点云中的数据点所关联的数据，点云数据的坐标例如可以是指数据点的(x,y,z)坐标。坐标位于候选框的坐标范围可以是指，数据点的(x,y,z)坐标属于候选框的坐标范围。候选框的坐标范围可以是指候选框所形成的长方体的范围。Point cloud data may refer to data associated with data points in the 3D point cloud, and the coordinates of the point cloud data may refer to (x, y, z) coordinates of the data points, for example. The fact that the coordinates are located in the coordinate range of the candidate box may mean that the (x, y, z) coordinates of the data point belong to the coordinate range of the candidate box. The coordinate range of the candidate box may refer to the range of the cuboid formed by the candidate box.
403、基于至少一个候选框分别对应的点云数量，确定最大点云数量对应的候选框为目标检测框。403. Based on the number of point clouds corresponding to at least one candidate frame, determine the candidate frame corresponding to the maximum number of point clouds as the target detection frame.
可选地，步骤403，可以包括：从至少一个候选框分别对应的点云数据中确定最大点云数量，将最大点云数量对应的候选框作为目标检测框。Optionally, step 403 may include: determining the maximum number of point clouds from the point cloud data corresponding to at least one candidate frame, and using the candidate frame corresponding to the maximum number of point clouds as the target detection frame.
最大点云数量可以为至少一个候选框分别对应的点云数量中的最大值。The maximum number of point clouds may be the maximum number of point clouds corresponding to at least one candidate frame.
本公开的技术方案中，为了从至少一个校正框和检测框中确定目标检测框，可以先将至少一个校正框和检测框进行统一管理，获得至少一个候选框。至少一个候选框可以分别获取点云数量，通过点云数量代表候选框的检测质量，进而使得候选框的质量评价数据更有效。因而，采用最大点云数量的候选框作为目标检测框，可以获得检测质量最高的目标检测框，实现目标检测框的高精度检测。In the technical solution of the present disclosure, in order to determine the target detection frame from at least one correction frame and detection frame, at least one correction frame and detection frame can be managed in a unified manner first to obtain at least one candidate frame. At least one candidate frame can obtain the number of point clouds respectively. The number of point clouds represents the detection quality of the candidate frame, thereby making the quality evaluation data of the candidate frame more effective. Therefore, by using the candidate frame with the largest number of point clouds as the target detection frame, the target detection frame with the highest detection quality can be obtained, and high-precision detection of the target detection frame can be achieved.
作为一种可选实施方式，将至少一个目标标注框分别变换为与检测框的几何信息相匹配的校正框，获得至少一个校正框，包括：As an optional implementation, converting at least one target annotation frame into a correction frame that matches the geometric information of the detection frame to obtain at least one correction frame includes:
按照检测框的原始比例对检测框进行尺寸调整，获得标准框，并提取3D点云中属于标准框的标准点云集合，标准框的尺寸是检测框的尺寸的预设倍数；Adjust the size of the detection frame according to the original proportion of the detection frame to obtain the standard frame, and extract the standard point cloud set belonging to the standard frame in the 3D point cloud. The size of the standard frame is a preset multiple of the size of the detection frame;
基于至少一个目标标注框分别对应的训练数据，提取至少一个目标标注框分别对应的标注点云集合，训练数据为参与训练的训练3D点云，训练数据关联标注框；Based on the training data corresponding to at least one target annotation frame, extract a set of annotation point clouds corresponding to at least one target annotation frame. The training data is the training 3D point cloud participating in the training, and the training data is associated with the annotation frame;
根据至少一个目标标注框分别对应的标注点云集合，结合标准点云集合，将至少一个目标标注框分别变换为校正框，获得至少一个校正框。According to the set of labeled point clouds respectively corresponding to at least one target labeling frame and combined with the set of standard point clouds, at least one target labeling frame is converted into a correction frame respectively, and at least one correction frame is obtained.
可选地，检测框的原始比例可以包括检测框的长宽比和宽高比。长宽比可以是指检测框的长度和宽度的比值。宽高比可以是指检测框的宽度和高度的比值。Optionally, the original proportion of the detection frame may include the aspect ratio and aspect ratio of the detection frame. The aspect ratio can refer to the ratio of the length and width of the detection frame. The aspect ratio can refer to the ratio of the width and height of the detection frame.
其中，按照检测框的原始比例对检测框进行尺寸调整，获得标准框，可以包括按照检测框的原始比例，对检测框的长、宽、高分别进行等比例的尺寸调整，获得调整后的标准框。标准框的尺寸是检测框的尺寸的预设倍数。预设倍数可以为预先设置的倍数，可以为一个根据使用需求设置常数，例如可以为1.2、1.5、2等常数。为了避免出现较大检测误差或失真，预设倍数的取值可以在一定数值范围内，例如大于第一数值且小于第二数值。第一数值和第二数值也可以根据使用需求设置。例如第一数值可以为1，第二数值可以为2，当然该取值方式仅是示例性的并不构成对本公开实施方式的具体限定。Among them, adjusting the size of the detection frame according to the original proportion of the detection frame to obtain the standard frame may include adjusting the length, width and height of the detection frame in equal proportions according to the original proportion of the detection frame to obtain the adjusted standard frame. The size of the standard frame is a preset multiple of the size of the detection frame. The preset multiple may be a preset multiple, or may be a constant set according to usage requirements, for example, it may be a constant such as 1.2, 1.5, 2, etc. In order to avoid large detection errors or distortions, the value of the preset multiple can be within a certain range of values, for example, greater than the first value and less than the second value. The first numerical value and the second numerical value can also be set according to usage requirements. For example, the first numerical value may be 1, and the second numerical value may be 2. Of course, this value method is only exemplary and does not constitute a specific limitation on the embodiments of the present disclosure.
按照检测框的原始比例，对检测框的长、宽、高分别进行等比例的尺寸调整，可以包括：根据检测框预设倍数，计算检测框的长和预设倍数的乘积，获得标准框的长，计算检测框的宽和预设倍数的乘积，获得标准框的宽，计算检测框的高和预设倍数的乘积，获得标准框的高。确定标准框的长、宽和高构成的标准框。According to the original proportion of the detection frame, adjust the length, width, and height of the detection frame in equal proportions. This may include: calculating the product of the length of the detection frame and the preset multiple according to the preset multiple of the detection frame to obtain the standard frame. length, calculate the product of the width of the detection frame and the preset multiple, and obtain the width of the standard frame. Calculate the product of the height of the detection frame and the preset multiple, and obtain the height of the standard frame. Determine the standard box formed by the length, width and height of the standard box.
按照检测框的原始比例对检测框进行尺寸调整，获得标准框之后，还包括，调整标准框的中心点以使得该中心点与检测框的中心点相同，调整标准框的方向以使得该方向与检测框的方向相同。Adjust the size of the detection frame according to the original proportion of the detection frame. After obtaining the standard frame, it also includes adjusting the center point of the standard frame so that the center point is the same as the center point of the detection frame, and adjusting the direction of the standard frame so that the direction is consistent with the center point of the detection frame. The detection frames have the same orientation.
检测框的原始比例和标准框的比例相同。进一步地，检测框的原始比例与标准框的比例相同，具体可以是指标准框的长宽比例和与检测框的长宽比例相同，以及标准框的宽高比例和检测框的宽高比例相同。进而标准框继承检测框的几何特性，实现等比例放大，获得的标准框在能包含更多点云数据的同时维持检测框的几何特性，使得标准框能够对目标对象进行更有效的包围。The original proportion of the detection frame is the same as that of the standard frame. Furthermore, the original proportion of the detection frame is the same as that of the standard frame, which specifically means that the length-to-width ratio of the standard frame is the same as the length-to-width ratio of the detection frame, and that the width-to-height ratio of the standard frame is the same as the width-to-height ratio of the detection frame. . Then, the standard frame inherits the geometric characteristics of the detection frame and achieves proportional enlargement. The obtained standard frame can contain more point cloud data while maintaining the geometric characteristics of the detection frame, allowing the standard frame to surround the target object more effectively.
本实施例中，将检测框调整为标准框的过程中，维持检测框和标准框的角度和位置一致，使得标准框仅是放大，而并不对角度和位置进行更新，确保标准框能够准确定位以实现对目标对象的更准确的包围。In this embodiment, during the process of adjusting the detection frame to the standard frame, the angle and position of the detection frame and the standard frame are kept consistent, so that the standard frame is only enlarged without updating the angle and position, ensuring that the standard frame can be accurately positioned to achieve more accurate encirclement of target objects.
可选地，训练集可以包括多个训练数据，训练数据可以为参与训练的训练3D点云。训练3D点云同样可以是指通过3D扫描设备扫描获得的点云。数据训练可以关联至少一个标注框。目标标注框可以对应训练数据。Optionally, the training set may include multiple training data, and the training data may be training 3D point clouds participating in the training. The training 3D point cloud can also refer to the point cloud obtained by scanning with a 3D scanning device. Data training can be associated with at least one annotation box. The target annotation box can correspond to the training data.
本实施例中，目标标注框的标注点云集合的获取步骤，可以包括：确定目标标注框的坐标范围，确定训练数据中多个训练点云数据分别对应的坐标，确定坐标位于目标标注框的坐标范围内的训练点云数据为目标标注框的标注点云集合。In this embodiment, the step of obtaining the annotation point cloud set of the target annotation frame may include: determining the coordinate range of the target annotation frame, determining the coordinates corresponding to multiple training point cloud data in the training data, and determining that the coordinates are located in the target annotation frame. The training point cloud data within the coordinate range is the annotation point cloud set of the target annotation box.
可选地，根据至少一个目标标注框分别对应的标注点云集合，结合标准点云集合，将至少一个目标标注框分别变换为校正框，获得至少一个校正框，可以包括：提取标注点云集合和标准点云集合之间的点云映射关系，获得至少一个目标标注框分别对应的点云映射关系，根据目标标注框点云映射关系将该目标标注框转换为校正框，获得至少一个目标标注框分别对应的校正框。Optionally, converting at least one target annotation frame into a correction frame respectively according to the annotation point cloud set corresponding to the at least one target annotation frame and combining with the standard point cloud set to obtain at least one correction frame may include: extracting the annotation point cloud set and the point cloud mapping relationship between the standard point cloud set, obtain the point cloud mapping relationship corresponding to at least one target labeling frame, convert the target labeling frame into a correction frame according to the point cloud mapping relationship of the target labeling frame, and obtain at least one target labeling The correction frames corresponding to the frames respectively.
进一步地，点云映射关系可以包括标注点云集合和标准点云集合之间的仿射变换矩阵。Further, the point cloud mapping relationship may include an affine transformation matrix between the annotated point cloud set and the standard point cloud set.
本公开的技术方案中，对目标标注框校正的过程中，可以先将检测框按照原有长宽比放大N倍，获得标准框，使得标准框包含更多的点云数据，能够有效包围目标对象。之后可以针对标准框的标准点云集合和至少一个目标标注框各自的标注点云集合，通过圈定各个框的点云集合，可以获取点之间的映射关系，进而利用点之间的映射实现框之间的映射。也即，将点云集合之间的点与点的映射关系，应用到框和框之间的映射，实现目标标注框到校正框的几何校正，获得与标准框的几何特征相匹配的校正框。而标准框保留了检测框的几何信息，进而校正框具备了包围目标对象的检测框的几何信息，校正框可以作为目标对象的一个检测结果，进而获得的至少一个校正框可以参与到目标检测框的选择，进一步提高了目标检测框的选择范围，获得具备更高检测精度的目标检测框。In the technical solution of the present disclosure, during the process of correcting the target labeling frame, the detection frame can be enlarged N times according to the original aspect ratio to obtain the standard frame, so that the standard frame contains more point cloud data and can effectively surround the target. object. Then, for the standard point cloud set of the standard frame and the annotated point cloud set of at least one target annotated frame, by delineating the point cloud set of each frame, the mapping relationship between the points can be obtained, and then the mapping between the points can be used to realize the frame mapping between. That is, the point-to-point mapping relationship between point cloud sets is applied to the mapping between boxes to achieve geometric correction from the target annotation box to the correction box, and obtain a correction box that matches the geometric features of the standard box. . The standard frame retains the geometric information of the detection frame, and the correction frame has the geometric information of the detection frame surrounding the target object. The correction frame can be used as a detection result of the target object, and at least one correction frame obtained can participate in the target detection frame. The selection further improves the selection range of target detection frames and obtains target detection frames with higher detection accuracy.
作为一个实施例，根据至少一个目标标注框分别对应的标注点云集合，结合标准点云集合，将至少一个目标标注框分别变换为校正框，获得至少一个校正框，包括：As an embodiment, according to the annotation point cloud set corresponding to at least one target annotation frame, combined with the standard point cloud set, at least one target annotation frame is converted into a correction frame, and at least one correction frame is obtained, including:
通过仿射变换算法，分别计算至少一个标注点云集合映射到标准点云集合的仿射变换矩阵；Calculate the affine transformation matrix mapping at least one labeled point cloud set to a standard point cloud set through the affine transformation algorithm;
根据标注点云集合的仿射变换矩阵对对应的目标标注框进行仿射变换，获得至少一个目标标注框分别对应的校正框。Perform affine transformation on the corresponding target annotation frame according to the affine transformation matrix of the annotation point cloud set, and obtain at least one correction frame corresponding to the target annotation frame.
可选地，仿射变换算法可以是指通过线性变换和平移确定的空间变换算法。通过仿射变换算法，分别计算至少一个标注点云集合映射到标准点云集合的仿射变换矩阵，可以包括：通过构造未知仿射变换矩阵；利用至少一个标注点云集合和标准点云集合，与未知仿射变换矩阵之间的矩阵转换关系，对每个标注点云集合转换到标准点云集合的未知仿射变换矩阵进行求解，获得至少一个标注点云集合分别对应的仿射变换矩阵。Alternatively, the affine transformation algorithm may refer to a spatial transformation algorithm determined by linear transformation and translation. Calculate the affine transformation matrix mapping at least one labeled point cloud set to a standard point cloud set through an affine transformation algorithm, which may include: constructing an unknown affine transformation matrix; using at least one labeled point cloud set and a standard point cloud set, The matrix conversion relationship with the unknown affine transformation matrix is to solve the unknown affine transformation matrix that converts each labeled point cloud collection to the standard point cloud collection, and obtain the affine transformation matrix corresponding to at least one labeled point cloud collection.
仿射变换矩阵可以包括矩阵A和附加的列b，其中，矩阵A可以用于线性变换，列b可以用于平移变换。本实施例中，可以使用标注点云集合和标准点云集合对仿射变换矩阵进行解算。The affine transformation matrix may include matrix A and an additional column b, where matrix A may be used for linear transformation and column b may be used for translation transformation. In this embodiment, the annotated point cloud set and the standard point cloud set can be used to solve the affine transformation matrix.
可选地，标注点云集合可以关联仿射变换矩阵和目标标注框。根据标注点云集合的仿射变换矩阵对对应的目标标注框进行仿射变换，可以包括：将标注点云集合的仿射变换矩阵，对该标注点云集合对应的目标标注框进行仿射变换计算，获得该目标标注框对应的校正框。校正框可以为目标标注框校正之后的框。Optionally, the annotated point cloud collection can be associated with an affine transformation matrix and a target annotation box. Performing affine transformation on the corresponding target annotation frame according to the affine transformation matrix of the annotated point cloud set may include: applying the affine transformation matrix of the annotated point cloud set to the target annotation frame corresponding to the annotated point cloud set. Calculate and obtain the correction frame corresponding to the target label frame. The correction frame can be the corrected frame for the target label frame.
本公开的技术方案中，可以通过仿射变换算法，分别计算至少一个标注点云集合映射到标准点云集合的仿射变换矩阵，通过仿射变换矩阵将对应的目标标注框进行仿射变换，获得该目标标注框的校正框。通过仿射变换可以准确表示点云集合之间的映射关系，进而利用仿射变换矩阵实现目标标注框的准确变换，使得标注点云集合与标准点云集合的映射关系应用到目标标注框的变换，提高目标标注框的变换精度和准确度。In the technical solution of the present disclosure, the affine transformation matrix mapping at least one annotated point cloud set to a standard point cloud set can be calculated through an affine transformation algorithm, and the corresponding target annotation frame can be affine transformed through the affine transformation matrix. Get the correction frame of the target label frame. The mapping relationship between point cloud sets can be accurately represented through affine transformation, and then the affine transformation matrix can be used to achieve accurate transformation of the target annotation frame, so that the mapping relationship between the annotation point cloud set and the standard point cloud set can be applied to the transformation of the target annotation frame. , improve the transformation precision and accuracy of the target annotation frame.
作为一个实施例，通过仿射变换算法，分别计算至少一个标注点云集合映射到标准点云集合的仿射变换矩阵，包括：As an embodiment, an affine transformation algorithm is used to calculate an affine transformation matrix mapping at least one labeled point cloud set to a standard point cloud set, including:
通过点云匹配算法，将标注点云集合与标准点云集合进行配对，获得至少一组点云映射对，点云映射对包括标注点云集合中的标注点云数据和标准点云集合中与标注点云数据具有映射关联的标准点云数据；Through the point cloud matching algorithm, the labeled point cloud set is paired with the standard point cloud set to obtain at least one set of point cloud mapping pairs. The point cloud mapping pair includes the labeled point cloud data in the labeled point cloud set and the standard point cloud set. Labeled point cloud data has standard point cloud data associated with mapping;
通过仿射变换算法，将至少一组点云映射对进行仿射计算，获得标注点云集合映射到标准点云集合的仿射变换矩阵。Through the affine transformation algorithm, affine calculation is performed on at least one set of point cloud mapping pairs to obtain an affine transformation matrix mapping the labeled point cloud set to the standard point cloud set.
可选地，通过点云匹配算法，将标注点云集合与标准点云集合进行配对，获得至少一组点云映射对，可以包括：将标注点云集合与标准点云集合输入点云匹配算法，通过点云匹配算法进行点云数据的配对，获得至少一组点云映射对。点云映射对可以包括标具备映射关系的标注点云数据和标准点云数据。Optionally, using a point cloud matching algorithm, pair the labeled point cloud set and the standard point cloud set to obtain at least one set of point cloud mapping pairs, which may include: inputting the labeled point cloud set and the standard point cloud set into the point cloud matching algorithm , pairing point cloud data through a point cloud matching algorithm to obtain at least one set of point cloud mapping pairs. Point cloud mapping pairs may include annotated point cloud data and standard point cloud data with mapping relationships.
点云匹配算法例如可以为ICP(Iterative Closest Point，点云配准算法)，NICP(Normal Iterative Closest Point，经典点云配准算法)等任意一种点云匹配算法，本实施例中对点云匹配算法的具体类型并不过多限定。The point cloud matching algorithm can be, for example, any point cloud matching algorithm such as ICP (Iterative Closest Point, point cloud registration algorithm), NICP (Normal Iterative Closest Point, classic point cloud registration algorithm), etc. In this embodiment, the point cloud matching algorithm is The specific type of matching algorithm is not too restrictive.
为了便于理解，假设标准点云集合使用Ppred表示，至少一个标注点云集合使用表示，其中，k为至少一个标注点云集合的数量或者至少一个目标标注框的数量，/>为目标标注框/>对应的仿射变换矩阵。For ease of understanding, it is assumed that the standard point cloud set is represented by Ppred , and at least one labeled point cloud set is represented by represents, where k is the number of at least one labeled point cloud set or the number of at least one target labeled box, /> Box the target/> The corresponding affine transformation matrix.
因此，通过点云匹配算法，计算映射到Ppred的仿射变换矩阵：Ti(i＝1,...,K)，Ti可以表示为：Therefore, through the point cloud matching algorithm, calculate Affine transformation matrix mapped to Ppred : Ti (i =1,...,K),Ti can be expressed as:
进一步地，获得标注点云集合映射到标准点云集合的仿射变换矩阵Ti之后，可以利用该仿射变换计算公式和仿射变换矩阵Ti，将标注点云集合对应的目标标注框进行仿射变换计算，获得该目标标注框/>对应的校正框/>放射变换计算公式为：Further, after obtaining the affine transformation matrix Ti that maps the labeled point cloud set to the standard point cloud set, the affine transformation calculation formula and the affine transformation matrix Ti can be used to label the target annotation frame corresponding to the labeled point cloud set. Perform affine transformation calculations to obtain the target label box/> Corresponding correction frame/> The radiation transformation calculation formula is:
本公开的技术方案中，使用点云匹配算法，将标注点云集合与标准点云集合进行配对，实现点与点之间的配对，获得至少一组点云映射对，通过至少一组点云映射对可以通过仿射计算，获得标注点云集合映射到标准点云集合的仿射变换矩阵。通过点云数据的配对可以明确标注点云结合与标准点云集合之间的点云数据的映射关系，进而实现以点为计算基础，实现更小维度的仿射计算，提高仿射变换精度。In the technical solution of the present disclosure, a point cloud matching algorithm is used to pair the annotated point cloud set with the standard point cloud set to achieve pairing between points and obtain at least one set of point cloud mapping pairs. Through at least one set of point cloud The mapping pair can be calculated through affine to obtain the affine transformation matrix that maps the labeled point cloud set to the standard point cloud set. Through the pairing of point cloud data, the mapping relationship between the point cloud data and the standard point cloud collection can be clearly marked, thereby realizing point-based calculations, achieving smaller-dimensional affine calculations, and improving the accuracy of affine transformation.
图5为本公开第三实施例的示意图，与前述实施例的不同之处在于，基于训练集中的多个标注框，筛选与检测框相匹配的至少一个目标标注框，包括：Figure 5 is a schematic diagram of the third embodiment of the present disclosure. The difference from the previous embodiment is that based on multiple annotation frames in the training set, at least one target annotation frame that matches the detection frame is screened, including:
501、将训练集中的多个标注框分组，获得至少一个标注框集合，标注框集合分别关联至少一个标注框以及模板标注框，模板标注框为从至少一个标注框中确定的标注框。501. Group multiple annotation boxes in the training set to obtain at least one annotation box set. The annotation box set is associated with at least one annotation box and a template annotation box. The template annotation box is an annotation box determined from at least one annotation box.
可选地，步骤501，可以包括：随机将多个标注框划分为至少一个标注框集合。或者步骤501还可以包括：通过哈希算法，将多个标注框划分为至少一个标注框集合。通过随机划分或者哈希划分可以实现多个标注框的快速划分。Optionally, step 501 may include: randomly dividing multiple annotation boxes into at least one annotation box set. Or step 501 may also include: dividing multiple annotation boxes into at least one annotation box set through a hash algorithm. Rapid division of multiple annotation boxes can be achieved through random division or hash division.
模板标注框可以是指从至少一个标注框中确定的标注质量最高的标注框。The template annotation box may refer to an annotation box with the highest annotation quality determined from at least one annotation box.
502、从至少一个标注框集合中，确定与检测框相匹配的K个目标标注框集合。502. Determine K target label frame sets that match the detection frame from at least one label frame set.
可选地，步骤502可以包括：从至少一个标注框集合中选择满足检测框的匹配条件的K个目标标注框集合。匹配条件可以包括标注框集合的匹配度大于或等于匹配度阈值或者标注框集合的匹配度属于匹配度最高的前K个目标匹配度。Optionally, step 502 may include: selecting K target annotation box sets that meet the matching conditions of the detection frame from at least one annotation box set. The matching conditions may include that the matching degree of the annotation box set is greater than or equal to the matching degree threshold or that the matching degree of the annotation box set belongs to the top K target matching degrees with the highest matching degree.
例如，可以确定至少一个标注框集合分别与检测框的匹配度，并确定匹配度大于或等于匹配度阈值的目标标注框集合，以获得与检测框相匹配的K个目标标注框集合。For example, the matching degree of at least one set of annotation boxes with the detection frame can be determined, and a set of target annotation boxes whose matching degree is greater than or equal to the matching degree threshold can be determined to obtain K sets of target annotation boxes that match the detection frame.
K为大于等于1的正整数，可以预先设置获得。例如，K可以设置为2。K is a positive integer greater than or equal to 1, which can be set in advance. For example, K can be set to 2.
503、确定K个目标标注框集合分别关联的模板标注框为至少一个目标标注框。503. Determine that the template label boxes associated with the K target label box sets are at least one target label box.
可选地，步骤503可以包括：根据至少一个标注框集合分别对应的模板标注框，依次读取K个目标标注框集合分别关联的模板标注框。Optionally, step 503 may include: sequentially reading the template annotation boxes associated with the K target annotation box sets according to the template annotation boxes respectively corresponding to at least one annotation box set.
在又一种可能的设计中，为了使得模板标注框至少一个目标标注框更有效，步骤503还可以包括：判断目标标注框集合的模板标注框是否满足使用条件，若是，则确定该模板标注框为目标标注框。In another possible design, in order to make at least one target label box of the template label box more effective, step 503 may also include: determining whether the template label box of the target label box set meets the usage conditions, and if so, determining the template label box Label the target box.
进一步地，判断目标标注框集合的模板标注框是否满足使用条件可以包括：计算目标标注框集合的模板标注框的尺寸与检测框的尺寸之间的尺寸比值，判断该尺寸比值是否小于比例阈值，若是，则确定该模板标注框满足使用条件，若否，则确定该模板标注框不满足使用条件。若模板标注框与检测框的尺寸比值差异较大，则可能会导致模板标注框相对检测框的过度变换，产生负面影响，此时可以舍弃该模板标注框，使得参与到最终目标检测框的选择的框更有效，减少无效的模板标注框参与到目标检测框的选择，提高选择效率和准确度。Further, determining whether the template annotation frame of the target annotation frame set meets the usage conditions may include: calculating the size ratio between the size of the template annotation frame of the target annotation frame set and the size of the detection frame, and determining whether the size ratio is less than the proportion threshold, If yes, it is determined that the template label box meets the usage conditions; if not, it is determined that the template label box does not meet the usage conditions. If the size ratio of the template annotation frame and the detection frame is very different, it may lead to excessive transformation of the template annotation frame relative to the detection frame, which will have a negative impact. In this case, the template annotation frame can be discarded to participate in the selection of the final target detection frame. The frames are more effective, reducing invalid template annotation frames to participate in the selection of target detection frames, and improving selection efficiency and accuracy.
进一步地，计算目标标注框集合的模板标注框的尺寸与检测框的尺寸之间的尺寸比值，可以是包括计算目标标注框集合的模板标注框与检测框之间的长度比值、宽度比值、高度比值以及体积比值中的至少一个。Further, calculating the size ratio between the size of the template labeling frame of the target labeling frame set and the size of the detection frame may include calculating the length ratio, width ratio, and height between the template labeling frame and the detection frame of the target labeling frame set. At least one of ratio and volume ratio.
本公开的技术方案中，通过将多个标注框分组，可以使得属于同一类型的标注框划分至同一集合中，获得的至少一个标注框集合可以分别代表一类标注框。进而从至少一个标注框集合中确定与检测框相匹配的K个目标标注框集合之后，可以获得与检测框相匹配的K个目标标注框集合，使用K个与检测框相匹配的目标标注框集合分别关联的模板标注框作为至少一个目标标注框。通过分类和匹配选择，可以实现快速获得至少一个目标标注框，避免将多个标注框分散参与到检测框的优化过程，通过分类的方式集中同类的标注框，并选择相应的模板标注框为目标标注框，有效提高了至少一个目标标注框的获取效率。In the technical solution of the present disclosure, by grouping multiple label boxes, label boxes belonging to the same type can be divided into the same set, and the obtained at least one label box set can each represent a type of label box. Then, after determining K target annotation frame sets that match the detection frame from at least one annotation frame set, K target annotation frame sets that match the detection frame can be obtained, and K target annotation frame sets that match the detection frame are used. Set respectively associated template label boxes as at least one target label box. Through classification and matching selection, it is possible to quickly obtain at least one target annotation frame, avoiding the scattering of multiple annotation frames into the optimization process of the detection frame, concentrating similar annotation frames through classification, and selecting the corresponding template annotation frame as the target Label box, which effectively improves the efficiency of obtaining at least one target label box.
作为一种可选实施方式，从至少一个标注框集合中，确定与检测框相匹配的K个目标标注框集合，包括：As an optional implementation, determine K target annotation box sets that match the detection frame from at least one annotation box set, including:
根据标注框集合关联的至少一个标注框，计算标注框集合与检测框对应的匹配度，获得至少一个标注框集合分别对应的匹配度；According to at least one annotation box associated with the annotation box set, calculate the matching degree corresponding to the annotation box set and the detection frame, and obtain the matching degree corresponding to at least one annotation box set;
基于至少一个标注框集合分别对应的匹配度，确定匹配度最大的K个目标标注框集合。Based on the matching degree corresponding to at least one annotation box set, K target annotation box sets with the largest matching degree are determined.
标注框集合与检测框之间的匹配度可以通过该标注框集合关联的至少一个标注框与检测框进行距离计算获得距离。The matching degree between the label box set and the detection frame can be obtained by calculating the distance between at least one label box associated with the label box set and the detection frame.
本实施例中，根据标注框集合关联的至少一个标注框，计算标注框集合与检测框对应的匹配度，可以包括：根据标注框集合关联的至少一个标注框，计算标注框集合与检测框对应的框距离，以框距离作为匹配度。距离可以是包括至少一个标注框与检测框之间的框距离。In this embodiment, calculating the matching degree between the annotation frame set and the detection frame based on at least one annotation frame associated with the annotation frame set may include: calculating the correspondence between the annotation frame set and the detection frame based on at least one annotation frame associated with the annotation frame set. The box distance is used as the matching degree. The distance may include a frame distance between at least one annotation frame and a detection frame.
其中，基于至少一个标注框集合分别对应的匹配度，确定匹配度最大的K个目标标注框集合，可以包括：将至少一个标注框集合分别对应的匹配度进行排序，从排序后的至少一个标注框集合中确定匹配度最大的K个目标标注框集合。Determining the K target label box sets with the largest matching degrees based on the matching degrees corresponding to at least one label box set may include: sorting the matching degrees corresponding to at least one label box set, and starting from the sorted at least one label box set. Determine the set of K target annotation boxes with the largest matching degree in the box set.
其中，匹配度最大的K个目标标注框集合可以是指至少一个标注框集合分别对应的匹配度中，匹配度的数值最大的前K个匹配度关联的标注框集合。The K target annotation box sets with the largest matching degrees may refer to the annotation box sets associated with the top K matching degrees that have the largest matching degrees among the matching degrees corresponding to at least one annotation box set.
本公开的技术方案中，根据标注框集合关联的至少一个标注框，可以计算该标注框集合与检测框的匹配度，进而将至少一个标注框集合与检测框的匹配过程进行详细计算，获得至少一个标注框集合分别对应的匹配度，通过基于至少一个标注框集合分别对应的匹配度可以实现K个目标标注框集合的直接、快速且准确的选择，提高K个目标标注框集合的选择效率和准确度。In the technical solution of the present disclosure, according to at least one annotation frame associated with the annotation frame set, the matching degree of the annotation frame set and the detection frame can be calculated, and then the matching process of the at least one annotation frame set and the detection frame is calculated in detail to obtain at least Based on the corresponding matching degree of at least one label box set, the direct, fast and accurate selection of K target label box sets can be achieved, improving the selection efficiency and accuracy of K target label box sets. Accuracy.
进一步地，在上述任一实施例的基础上，根据标注框集合关联的至少一个标注框，计算标注框集合与检测框对应的匹配度，包括：Further, based on any of the above embodiments, calculating the matching degree corresponding to the annotation frame set and the detection frame according to at least one annotation frame associated with the annotation frame set, including:
基于标注框集合关联的至少一个标注框，计算标注框与检测框的框距离，获得至少一个标注框分别对应的框距离；Based on at least one annotation frame associated with the annotation frame set, calculate the frame distance between the annotation frame and the detection frame, and obtain the frame distance corresponding to at least one annotation frame;
将至少一个标注框分别对应的框距离进行均值计算，以将获得的均值距离作为标注框集合与检测框对应的匹配度。Calculate the mean value of the frame distance corresponding to at least one label box, and use the obtained mean distance as the matching degree between the label box set and the detection frame.
可选地，框距离可以是指通过至少一个标注框和检测框进行距离计算获得的距离。标注框和检测框的框距离的计算步骤可以包括：确定标注框的位置坐标和检测框的位置坐标，根据标注框的位置坐标和检测框的位置坐标，计算模板标注框和检测框的框距离。关于坐标距离计算公式可以参考下列实施例的描述。Optionally, the frame distance may refer to the distance obtained by performing distance calculation between at least one label frame and the detection frame. The step of calculating the frame distance between the labeling frame and the detection frame may include: determining the position coordinates of the labeling frame and the position coordinates of the detection frame, and calculating the frame distance between the template labeling frame and the detection frame based on the position coordinates of the labeling frame and the position coordinates of the detection frame. . Regarding the coordinate distance calculation formula, please refer to the description of the following embodiments.
或者，标注框和检测框的框距离的计算步骤还可以包括：模板标注框和检测框的框距离的计算步骤可以包括：确定标注框的特征向量和检测框的特征向量，根据标注框的特征向量和检测框的特征向量，计算模板标注框和检测框的框距离。关于向量距离计算公式可以参考下列实施例的描述。Alternatively, the step of calculating the frame distance between the annotation frame and the detection frame may also include: the step of calculating the frame distance between the template annotation frame and the detection frame may include: determining the feature vector of the annotation frame and the feature vector of the detection frame, and based on the characteristics of the annotation frame Vector and the feature vector of the detection frame, calculate the frame distance between the template annotation frame and the detection frame. Regarding the vector distance calculation formula, please refer to the description of the following embodiments.
本实施例中，将至少一个标注框分别对应的框距离进行均值计算，可以包括：将至少一个标注框分别对应的框距离相加，获得距离和，并确定至少一个标注框的框数量，计算距离和与框数量的商，该商即可以确定为均值距离。In this embodiment, calculating the mean value of the frame distances corresponding to at least one annotation frame may include: adding the frame distances corresponding to at least one annotation frame to obtain the distance sum, and determining the number of frames of at least one annotation frame, calculating The quotient of the distance and the number of boxes can be determined as the mean distance.
本公开的技术方案中，利用标注框集合关联的至少一个标注框，计算该至少一个标注框分别与检测框的框距离，使得至少一个标注框参与到计算过程中。之后，将至少一个标注框分别对应的框距离进行均值计算，以获得的均值距离作为标注框与检测框的匹配度，通过均值计算方法，可以使得标注框集合中的至少一个标注框均参到与检测框的匹配过程中，可以使得匹配度的表示更均衡，进而利用至少一个标注框集合分别对应的匹配度进行K个目标标注框集合的选择时，选择对象更平稳，选择准确度更高。In the technical solution of the present disclosure, at least one annotation frame associated with the annotation frame set is used to calculate the frame distance between the at least one annotation frame and the detection frame, so that at least one annotation frame participates in the calculation process. After that, the frame distance corresponding to at least one annotation frame is averaged, and the obtained average distance is used as the matching degree between the annotation frame and the detection frame. Through the average calculation method, at least one annotation frame in the annotation frame set can be included in the mean distance. During the matching process with the detection frame, the expression of the matching degree can be made more balanced, and then when the matching degree corresponding to at least one annotation box set is used to select K target annotation box sets, the selection object is smoother and the selection accuracy is higher. .
进一步地，在上述任一实施例的基础上，根据标注框集合关联的至少一个标注框，计算标注框集合与检测框对应的匹配度，包括：Further, based on any of the above embodiments, calculating the matching degree corresponding to the annotation frame set and the detection frame according to at least one annotation frame associated with the annotation frame set, including:
确定至少一个标注框集合分别对应的模板标注框；Determine the template label box corresponding to at least one label box set;
计算至少一个标注框集合的模板标准框分别与检测框的框距离；Calculate the frame distance between the template standard frame of at least one annotation frame set and the detection frame;
确定标注框集合的框距离为标注框集合与检测框的匹配度，获得至少一个标注框集合分别对应的匹配度。Determine the frame distance of the label box set as the matching degree between the label box set and the detection frame, and obtain the matching degree corresponding to at least one label box set.
可选地，模板标注框可以为从标注框集合中确定的标注框。Optionally, the template callout box may be a callout box determined from a callout box collection.
可选地，模板标注框和检测框的框距离的计算步骤可以包括：确定模板标注框的位置坐标和检测框的位置坐标，根据模板标注框的位置坐标和检测框的位置坐标，计算模板标注框和检测框的框距离。Optionally, the step of calculating the frame distance between the template labeling frame and the detection frame may include: determining the position coordinates of the template labeling frame and the position coordinates of the detection frame, and calculating the template labeling based on the position coordinates of the template labeling frame and the position coordinates of the detection frame. The distance between the box and the detection box.
进一步地，可以根据坐标距离公式，计算模板标注框的位置坐标和检测框的位置坐标之间的坐标距离，并确定该坐标距离为框距离。坐标距离公式例如可以为：其中，(x1，y1，z1)为模板标注框的位置坐标，(x2，y2，z2)为检测框的位置坐标。Further, the coordinate distance between the position coordinates of the template annotation frame and the position coordinates of the detection frame can be calculated according to the coordinate distance formula, and the coordinate distance can be determined as the frame distance. The coordinate distance formula can be, for example: Among them, (x1 , y1 , z1 ) are the position coordinates of the template annotation frame, and (x2 , y2 , z2 ) are the position coordinates of the detection frame.
可选地，模板标注框和检测框的框距离的计算步骤可以包括：确定模板标注框的特征向量和检测框的特征向量，根据模板标注框的特征向量和检测框的特征向量，计算模板标注框和检测框的框距离。Optionally, the step of calculating the frame distance between the template annotation box and the detection frame may include: determining the feature vector of the template annotation box and the feature vector of the detection frame, and calculating the template annotation based on the feature vector of the template annotation box and the feature vector of the detection frame. The distance between the box and the detection box.
进一步地，可以根据向量距离公式，计算模板标注框的位置坐标和检测框的位置坐标之间的向量距离，并确定该向量距离为框距离。Further, the vector distance between the position coordinates of the template annotation frame and the position coordinates of the detection frame can be calculated according to the vector distance formula, and the vector distance can be determined as the frame distance.
向量距离公式例如可以为欧几里得距离公式、马氏距离公式、曼哈顿距离公式等，本实施例中对此并不过多限定。The vector distance formula may be, for example, a Euclidean distance formula, a Mahalanobis distance formula, a Manhattan distance formula, etc., which are not too limited in this embodiment.
本公开的技术方案中，利用标注框集合的模板标注框与检测框进行距离计算，获得的框距离即可以作为标注框集合与检测框的匹配度。而模板标注框即为最能代表标注框集合的标注框，将模板标注框与检测框的框距离作为匹配度，可以使得该匹配度更准确，更能体现标注框集合与检测框的匹配效果。In the technical solution of the present disclosure, distance calculation is performed between the template annotation frame and the detection frame of the annotation frame set, and the obtained frame distance can be used as the matching degree between the annotation frame set and the detection frame. The template annotation box is the annotation box that best represents the annotation box set. Using the frame distance between the template annotation box and the detection frame as the matching degree can make the matching degree more accurate and better reflect the matching effect of the annotation box set and the detection frame. .
作为一种可选实施方式，将训练集中的多个标注框分组，获得至少一个标注框集合之后，还包括：As an optional implementation, after grouping multiple annotation boxes in the training set and obtaining at least one annotation box set, the method further includes:
基于标注框集合对应的至少一个标注框，提取标注框集合对应的模板标注框，获得至少一个标注框集合分别对应的模板标注框。Based on at least one label box corresponding to the label box set, extract the template label box corresponding to the label box set, and obtain the template label box corresponding to at least one label box set.
可选地，基于标注框集合对应的至少一个标注框，提取标注框集合对应的模板标注框，可以包括：从标注框集合对应的至少一个标注框中最随机确定标注框为模板标注框。Optionally, extracting a template label box corresponding to the label box set based on at least one label box corresponding to the label box set may include: most randomly determining the label box as the template label box from at least one label box corresponding to the label box set.
此外，基于标注框集合对应的至少一个标注框，提取标注框集合对应的模板标注框，可以包括：基于标注框集合对应的至少一个标注框，提取标注框集合对应的模板标注框，可以包括：从标注框集合对应的至少一个标注框中选择质量评价最高的标注框作为模板标注框。In addition, extracting a template annotation box corresponding to the annotation box set based on at least one annotation box corresponding to the annotation box set may include: extracting a template annotation box corresponding to the annotation box set based on at least one annotation box corresponding to the annotation box set may include: Select the label box with the highest quality evaluation from at least one label box corresponding to the label box set as the template label box.
本公开的技术方案中，可以从标注框集合对应的至少一个标注框中提取标注框集合对应的模板标注框，使得模板标注框来源于标注框集合，更能体现标注框集合内至少一个标注框的共性，通过模板标注框可以有效对标注框集合进行表示。进而将K个目标标注框集合分别关联的模板标注框作为至少一个目标标注框，可以以更小的计算复杂度实现更有效的目标标注框的选择。In the technical solution of the present disclosure, the template label box corresponding to the label box set can be extracted from at least one label box corresponding to the label box set, so that the template label box originates from the label box set and can better reflect at least one label box in the label box set. The commonality of the template label box can effectively represent the label box set. Then, the template labeling boxes associated with the K target labeling box sets are used as at least one target labeling box, so that more effective target labeling frame selection can be achieved with less computational complexity.
在一种可能的设计中，基于标注框集合对应的至少一个标注框，提取标注框集合对应的模板标注框，包括：In one possible design, based on at least one label box corresponding to the label box set, the template label box corresponding to the label box set is extracted, including:
对标注框集合对应的至少一个标注框分别进行质量评价，获得至少一个标注框分别对应的质量评价数据；Perform quality evaluation on at least one annotation box corresponding to the annotation box set, and obtain quality evaluation data corresponding to at least one annotation box respectively;
根据至少一个标注框分别对应的质量评价数据，选择质量评价数据最大的标注框为标注框集合的模板标注框。According to the quality evaluation data corresponding to at least one label box, the label box with the largest quality evaluation data is selected as the template label box of the label box set.
可选地，对标注框集合对应的至少一个标注框分别进行质量评价，获得至少一个标注框分别对应的质量评价数据，可以包括：显示标注框集合对应的至少一个标注框的评分页面，响应于用户针对评分页面触发针对至少一个标注框分别执行的打分操作，获得至少一个标注框分别对应的质量分数，以将至少一个标注框分别对应的质量分数作为质量评价数据。Optionally, performing a quality evaluation on at least one annotation box corresponding to the annotation box set and obtaining quality evaluation data corresponding to at least one annotation box may include: displaying a rating page of at least one annotation box corresponding to the annotation box set, and responding to The user triggers a scoring operation for at least one annotation box on the rating page to obtain a quality score corresponding to at least one annotation box, and uses the quality score corresponding to at least one annotation box as quality evaluation data.
本实施例中，根据至少一个标注框分别对应的质量评价数据，选择质量评价数据最大的标注框为标注框集合的模板标注框，可以包括：将至少一个标注框分别对应的质量评价数据进行排序，并从排序后的至少一个标注框中选择质量评价数据最大的标注框作为模板标注框。In this embodiment, according to the quality evaluation data corresponding to at least one label box, selecting the label box with the largest quality evaluation data as the template label box of the label box set may include: sorting the quality evaluation data corresponding to at least one label box. , and select the annotation box with the largest quality evaluation data from at least one annotation box after sorting as the template annotation box.
本公开的技术方案中，为标注框集合提取模板标注框的过程中，可以质量评价的方式，从至少一个标注框中确定质量评价数据最大的标注框作为模板标注框，以使得模板标注框的质量评价最高。进而将质量评价最高的标注框作为标注框集合的模板标注框，实现各个标注框集合的模板标注框的准确且有效的获取。In the technical solution of the present disclosure, in the process of extracting the template annotation box for the annotation box set, the annotation box with the largest quality evaluation data from at least one annotation box can be determined as the template annotation box in a quality evaluation manner, so that the template annotation box can be Top quality rating. Then, the annotation box with the highest quality evaluation is used as the template annotation box of the annotation box set to achieve accurate and effective acquisition of the template annotation box of each annotation box set.
为了实现准确且有效的质量评价，进一步地，对标注框集合对应的至少一个标注框分别进行质量评价，获得至少一个标注框分别对应的质量评价数据，可以包括：In order to achieve accurate and effective quality evaluation, further, performing quality evaluation on at least one annotation box corresponding to the annotation box set and obtaining quality evaluation data corresponding to at least one annotation box respectively may include:
根据标注框集合对应的至少一个标注框，确定至少一个标注框在对应训练数据包围的点云数据的点云数量；Determine the number of point clouds in the point cloud data surrounded by the at least one annotation frame in the corresponding training data according to at least one annotation frame corresponding to the annotation frame set;
将标注框的在对应训练数据的点云数据作为标注框的质量评价数据，获得至少一个标注框分别对应的质量评价数据。The point cloud data of the corresponding training data of the labeling frame is used as the quality evaluation data of the labeling frame, and the quality evaluation data corresponding to at least one labeling frame is obtained.
可选地，根据标注框集合对应的至少一个标注框，确定至少一个标注框在对应训练数据包围的点云数据的点云数量，可以包括：确定标注框在训练数据对应的坐标范围，以及训练数据中多个点云数据分别对应的点云坐标，确定点云坐标位于标注框的坐标范围内的有效点云数据，并确定有效点云数据的点云数量，以获得标注框在训练数据包围的点云数据的点云数量。Optionally, determining the number of point clouds in the point cloud data surrounded by the at least one annotation frame in the corresponding training data according to at least one annotation frame corresponding to the annotation frame set may include: determining the coordinate range of the annotation frame corresponding to the training data, and training The point cloud coordinates corresponding to the multiple point cloud data in the data are determined to determine the valid point cloud data whose point cloud coordinates are within the coordinate range of the annotation frame, and the number of point clouds of the valid point cloud data is determined to obtain the annotation frame surrounded by the training data. The number of point clouds of point cloud data.
其中，将标注框的在对应训练数据的点云数据作为标注框的质量评价数据可以是指确定标注框的在对应训练数据的点云数据为该标注框的质量评价数据。Wherein, using the point cloud data of the annotation frame in the corresponding training data as the quality evaluation data of the annotation frame may refer to determining that the point cloud data of the annotation frame in the corresponding training data is the quality evaluation data of the annotation frame.
点云数量可以为标注框在对应训练数据所包围的点云数据的点云数量。The number of point clouds may be the number of point clouds of point cloud data surrounded by the labeling frame in the corresponding training data.
本公开的技术方案中，可以将标注框在训练数据所包围的点云数据的点云数量作为标注框的质量评价数据。包围的点云数据的点云数量越大，代表标注框对对象的标注效果越高，该标注框的标注质量也就越高，因此，将点云数量作为标注框的质量评价数据，有效提高了标注框的质量评价效果。In the technical solution of the present disclosure, the number of point clouds of the point cloud data surrounded by the annotation frame in the training data can be used as the quality evaluation data of the annotation frame. The greater the number of point clouds in the surrounding point cloud data, the higher the labeling effect of the labeling frame on the object, and the higher the labeling quality of the labeling frame. Therefore, using the number of point clouds as the quality evaluation data of the labeling frame can effectively improve The quality evaluation effect of the annotation box is improved.
进一步地，在上述任一实施例的基础上，将训练集中的多个标注框分组，获得至少一个标注框集合，包括：Further, based on any of the above embodiments, group multiple annotation boxes in the training set to obtain at least one annotation box set, including:
基于训练集中的多个标注框，提取标注框的特征向量，获得多个标注框分别对应的特征向量；Based on multiple annotation boxes in the training set, extract the feature vectors of the annotation boxes and obtain the feature vectors corresponding to the multiple annotation boxes;
根据聚类算法，将多个标注框分别对应的特征向量进行聚类，获得至少一个标注框集合。According to the clustering algorithm, the feature vectors corresponding to multiple label boxes are clustered to obtain at least one label box set.
可选地，提取标注框的特征向量可以包括：根据标注框的几何信息，提取标注框的特征向量。Optionally, extracting the feature vector of the annotation box may include: extracting the feature vector of the annotation box according to the geometric information of the annotation box.
进一步地，标注框的几何信息可以表示为：b＝[x,y,z,l,w,h,θ]。Furthermore, the geometric information of the annotation box can be expressed as: b=[x, y, z, l, w, h, θ].
其中，(x,y,z)为标注框的位置坐标，l为标注框的长度、w为标注框的宽度，θ的方向，例如航向角。可以根据标注框的几何信息，结合向量转化公式，将几何信息转换为特征向量，以降低信息维度。Among them, (x, y, z) are the position coordinates of the labeling box, l is the length of the labeling box, w is the width of the labeling box, and the direction of θ, such as the heading angle. According to the geometric information of the annotation box and the vector conversion formula, the geometric information can be converted into feature vectors to reduce the information dimension.
向量转化公式可以表示为：The vector conversion formula can be expressed as:
本实施例中的聚类算法例如可以为K-means(K-均值)聚类算法等任意一种聚类算法，本实施例中对此并不过多限定。The clustering algorithm in this embodiment can be, for example, any clustering algorithm such as K-means (K-means) clustering algorithm, which is not too limited in this embodiment.
本公开的技术方案中，可以基于训练集中的多个标注框，提取标注框的特征向量，使用特征向量对标注框进行聚类计算，以获得至少一个标注框集合。通过可以准确表示各个标注框的特征信息，进而利用特征向量进行聚类，可以实现快速且精准的分组。In the technical solution of the present disclosure, feature vectors of the annotation boxes can be extracted based on multiple annotation boxes in the training set, and the feature vectors can be used to perform clustering calculations on the annotation boxes to obtain at least one annotation box set. By accurately representing the feature information of each annotation box, and then using feature vectors for clustering, fast and accurate grouping can be achieved.
进一步地，在上述任一实施例的基础上，根据至少一个目标标注框对检测框进行校验处理，获得目标检测框之后，还包括：Further, on the basis of any of the above embodiments, the detection frame is verified according to at least one target annotation frame. After obtaining the target detection frame, it also includes:
根据3D点云和目标检测框，生成目标对象的检测结果；Generate target object detection results based on the 3D point cloud and target detection frame;
输出检测结果。Output the detection results.
可选地，根据3D点云和目标检测框，生成目标对象的检测结果，可以包括：将目标检测框在3D点云中渲染，获得目标对象的检测页面，以将该检测页面作为目标对象的检测结果。Optionally, generating a detection result of the target object based on the 3D point cloud and the target detection frame may include: rendering the target detection frame in the 3D point cloud, obtaining a detection page of the target object, and using the detection page as the target object's detection page. Test results.
以辅助驾驶领域为例，输出检测结果，可以包括：将检测结果发送至车载显示屏幕中，并控制车载显示屏幕显示该检测结果。Taking the field of assisted driving as an example, outputting detection results may include: sending the detection results to the vehicle-mounted display screen, and controlling the vehicle-mounted display screen to display the detection results.
本实施例中，可以根据3D点云和目标检测框，生成目标对象的检测结果，以输出该检测结果。通过检测结果的输出，可以实现目标对象的检测交互，提高用户体验。In this embodiment, the detection result of the target object can be generated based on the 3D point cloud and the target detection frame to output the detection result. Through the output of detection results, the detection interaction of the target object can be realized and the user experience can be improved.
为了便于理解本公开的技术方案，以车辆作为目标对象对本公开的技术方案的应用场景进行详细说明。In order to facilitate understanding of the technical solution of the present disclosure, the application scenarios of the technical solution of the present disclosure are described in detail with vehicles as the target object.
参考图6，对于3D点云601，可以从3D点云601中检测包围车辆的检测框602。Referring to FIG. 6 , for the 3D point cloud 601 , a detection frame 602 surrounding the vehicle may be detected from the 3D point cloud 601 .
之后，可以从训练集中确定与车辆类别相同的多个标注框。多个标注框可以表示为：其中，n为表示为标注框集合中第n个标注框，n的取值为1-N，N为多个标注框的标注框数量，c为车辆类别标识。Afterwards, multiple annotation boxes with the same vehicle category can be determined from the training set. Multiple callout boxes can be expressed as: Among them, n is represented as the nth label box in the label box set, the value of n is 1-N, N is the number of label boxes of multiple label boxes, and c is the vehicle category identifier.
根据本公开的技术方案，可以将多个标注框进行聚类，获得至少一个标注框集合其中，M为模板标注框的数量。每个标注框集合关联至少一个标注框。可以为每个标注框集合确定相应的模板标注框，获得至少一个标注框集合分别对应的模板标注框，具体如图6所示的模板标注框603。According to the technical solution of the present disclosure, multiple annotation boxes can be clustered to obtain at least one annotation box set. Among them, M is the number of template annotation boxes. Each set of callout boxes is associated with at least one callout box. The corresponding template labeling box can be determined for each labeling box set, and the template labeling box corresponding to at least one labeling box set is obtained, specifically the template labeling box 603 shown in Figure 6 .
之后，进行模板筛选，具体的筛选过程可以包括：计算至少一个模板标注框分别与检测框的框距离，并根据确定标注框集合的框距离为标注框集合与检测框的匹配度，获得至少一个标注框集合分别对应的匹配度。之后，利用基于至少一个标注框集合分别对应的匹配度，确定匹配度最大的K个目标标注框集合，以将K个目标标注框集合作为至少一个目标标注框。K个目标标注框集合分别关联的模板标注框可以表示为：K为至少一个目标标注框的数量，j为1-K的正整数。假设获得的目标标注框如604所示。Afterwards, template screening is performed. The specific screening process may include: calculating the frame distance between at least one template annotation frame and the detection frame, and determining the matching degree between the annotation frame set and the detection frame based on the frame distance of the annotation frame set to obtain at least one The corresponding matching degree of the label box set. Afterwards, K target label box sets with the largest matching degrees are determined using matching degrees corresponding to at least one label box set, so that the K target label box sets are used as at least one target label box. The template label boxes associated with the K target label box sets can be expressed as: K is the number of at least one target annotation box, and j is a positive integer from 1-K. Assume that the obtained target label box is shown in 604.
之后，通过点云匹配算法和仿射变换算法，将至少一个目标标注框进行仿射变换，获得至少一个目标标注框分别对应的校正框。最后从至少一个校正框和预测框中确定目标检测框605。具体的校正过程可以参考前述实施例的描述，在此不再赘述。Afterwards, the point cloud matching algorithm and the affine transformation algorithm are used to perform affine transformation on at least one target annotation frame, and a correction frame corresponding to at least one target annotation frame is obtained. Finally, the target detection frame 605 is determined from at least one correction frame and prediction frame. For the specific correction process, reference can be made to the description of the foregoing embodiments and will not be described again here.
图7为本公开第四实施例的示意图，参考图7所示的一种基于点云的目标检测装置，该装置700可以包括下列单元：Figure 7 is a schematic diagram of a fourth embodiment of the present disclosure. Referring to a point cloud-based target detection device shown in Figure 7, the device 700 may include the following units:
获取单元701、用于获取从3D点云中获得的检测框，检测框为包围3D点云中目标对象的检测框；The acquisition unit 701 is used to acquire a detection frame obtained from the 3D point cloud, where the detection frame is a detection frame surrounding the target object in the 3D point cloud;
筛选单元702、用于基于训练集中的多个标注框，筛选与检测框相匹配的至少一个目标标注框；The screening unit 702 is configured to filter at least one target annotation frame that matches the detection frame based on the multiple annotation frames in the training set;
校正单元703、用于根据至少一个目标标注框对检测框进行校验处理，获得目标检测框。The correction unit 703 is configured to perform verification processing on the detection frame according to at least one target annotation frame to obtain the target detection frame.
作为一个实施例，校正单元，包括：As an embodiment, the correction unit includes:
几何校正模块，用于将至少一个目标标注框分别变换为与检测框的几何信息相匹配的校正框，获得至少一个校正框；A geometric correction module, configured to transform at least one target annotation frame into a correction frame that matches the geometric information of the detection frame, to obtain at least one correction frame;
目标确定模块，用于基于至少一个校正框和检测框，确定目标检测框。The target determination module is used to determine the target detection frame based on at least one correction frame and a detection frame.
作为又一个实施例，目标确定模块，包括：As another embodiment, the target determination module includes:
候选确定子模块，用于基于至少一个校正框和检测框，确定至少一个候选框；A candidate determination submodule, used to determine at least one candidate frame based on at least one correction frame and detection frame;
数量确定子模块，用于确定至少一个候选框分别在3D点云对应的点云数量，点云数量为候选框在3D点云包含的点云数据的数量；The quantity determination submodule is used to determine the number of point clouds corresponding to at least one candidate frame in the 3D point cloud. The number of point clouds is the number of point cloud data contained in the 3D point cloud of the candidate frame;
目标确定子模块，用于基于至少一个候选框分别对应的点云数量，确定最大点云数量对应的候选框为目标检测框。The target determination submodule is used to determine the candidate frame corresponding to the maximum number of point clouds as the target detection frame based on the number of point clouds corresponding to at least one candidate frame.
作为又一个实施例，几何校正模块，包括：As another embodiment, the geometric correction module includes:
标准提取子模块，用于将检测框按照原有长宽比和宽高比放大N倍，获得标准框，并提取3D点云中属于标准框的标准点云集合；The standard extraction submodule is used to enlarge the detection frame N times according to the original aspect ratio and aspect ratio, obtain the standard frame, and extract the standard point cloud set belonging to the standard frame in the 3D point cloud;
标注提取子模块，用于基于至少一个目标标注框分别对应的训练数据，提取至少一个目标标注框分别对应的标注点云集合，训练数据为参与训练的训练3D点云，训练数据关联标注框；The annotation extraction submodule is used to extract the annotation point cloud set corresponding to at least one target annotation frame based on the training data corresponding to at least one target annotation frame. The training data is the training 3D point cloud participating in the training, and the training data is associated with the annotation frame;
标注校正子模块，用于根据至少一个目标标注框分别对应的标注点云集合，结合标准点云集合，将至少一个目标标注框分别变换为校正框，获得至少一个校正框。The annotation correction sub-module is used to transform at least one target annotation frame into a correction frame respectively according to the annotation point cloud set corresponding to at least one target annotation frame and combined with the standard point cloud set to obtain at least one correction frame.
作为又一个实施例，标注校正子模块，具体用于：As another embodiment, the syndrome labeling module is specifically used for:
通过仿射变换算法，分别计算至少一个标注点云集合映射到标准点云集合的仿射变换矩阵；Calculate the affine transformation matrix mapping at least one labeled point cloud set to a standard point cloud set through the affine transformation algorithm;
根据标注点云集合的仿射变换矩阵对对应的目标标注框进行仿射变换，获得至少一个目标标注框分别对应的校正框。Perform affine transformation on the corresponding target annotation frame according to the affine transformation matrix of the annotation point cloud set, and obtain at least one correction frame corresponding to the target annotation frame.
作为又一个实施例，标注校正子模块，具体用于：As another embodiment, the syndrome labeling module is specifically used for:
通过点云匹配算法，将标注点云集合与标准点云集合进行配对，获得至少一组点云映射对，点云映射对包括标注点云集合中的标注点云数据和标准点云集合中与标注点云数据具有映射关联的标准点云数据；Through the point cloud matching algorithm, the labeled point cloud set is paired with the standard point cloud set to obtain at least one set of point cloud mapping pairs. The point cloud mapping pair includes the labeled point cloud data in the labeled point cloud set and the standard point cloud set. Labeled point cloud data has standard point cloud data associated with mapping;
通过仿射变换算法，将至少一组点云映射对进行仿射计算，获得标注点云集合映射到标准点云集合的仿射变换矩阵。Through the affine transformation algorithm, affine calculation is performed on at least one set of point cloud mapping pairs to obtain an affine transformation matrix mapping the labeled point cloud set to the standard point cloud set.
作为又一个实施例，筛选单元，包括：As another embodiment, the screening unit includes:
标注分组模块，用于将训练集中的多个标注框分组，获得至少一个标注框集合，标注框集合分别关联至少一个标注框以及模板标注框，模板标注框为从至少一个标注框中确定的标注框；The annotation grouping module is used to group multiple annotation boxes in the training set to obtain at least one annotation box set. The annotation box set is respectively associated with at least one annotation box and a template annotation box. The template annotation box is an annotation determined from at least one annotation box. frame;
目标匹配模块，用于从至少一个标注框集合中，确定与检测框相匹配的K个目标标注框集合；A target matching module, used to determine K target annotation frame sets that match the detection frame from at least one annotation frame set;
目标确定模块，与确定K个目标标注框集合分别关联的模板标注框为至少一个目标标注框。The target determination module determines that the template label box associated with the K target label box sets is at least one target label box.
作为又一个实施例，目标匹配模块，包括：As another embodiment, the target matching module includes:
第一计算子模块，用于根据标注框集合关联的至少一个标注框，计算标注框集合与检测框对应的匹配度，获得至少一个标注框集合分别对应的匹配度；The first calculation submodule is used to calculate the matching degree corresponding to the annotation box set and the detection frame according to at least one annotation frame associated with the annotation frame set, and obtain the matching degree corresponding to at least one annotation frame set;
第一匹配子模块，用于基于至少一个标注框集合分别对应的匹配度，确定匹配度最大的K个目标标注框集合。The first matching sub-module is used to determine the K target label box sets with the largest matching degrees based on the matching degrees corresponding to at least one label box set.
作为又一个实施例，第一计算子模块，具体用于：As another embodiment, the first calculation sub-module is specifically used for:
基于标注框集合关联的至少一个标注框，计算标注框与检测框的框距离，获得至少一个标注框分别对应的框距离；Based on at least one annotation frame associated with the annotation frame set, calculate the frame distance between the annotation frame and the detection frame, and obtain the frame distance corresponding to at least one annotation frame;
将至少一个标注框分别对应的框距离进行均值计算，以将获得的均值距离作为标注框集合与检测框对应的匹配度。Calculate the mean value of the frame distance corresponding to at least one label box, and use the obtained mean distance as the matching degree between the label box set and the detection frame.
作为又一个实施例，第一计算子模块，具体用于：As another embodiment, the first calculation sub-module is specifically used for:
确定至少一个标注框集合分别对应的模板标注框；Determine the template label box corresponding to at least one label box set;
计算至少一个标注框集合的模板标准框分别与检测框的框距离；Calculate the frame distance between the template standard frame of at least one annotation frame set and the detection frame;
确定标注框集合的框距离为标注框集合与检测框的匹配度，获得至少一个标注框集合分别对应的匹配度。Determine the frame distance of the label box set as the matching degree between the label box set and the detection frame, and obtain the matching degree corresponding to at least one label box set.
作为又一个实施例，还包括：As yet another embodiment, it also includes:
模板获取单元，用于基于标注框集合对应的至少一个标注框，提取标注框集合对应的模板标注框，获得至少一个标注框集合分别对应的模板标注框。The template acquisition unit is configured to extract the template annotation box corresponding to the annotation box set based on at least one annotation box corresponding to the annotation box set, and obtain the template annotation box corresponding to the at least one annotation box set.
作为又一个实施例，模板获取单元，包括：As another embodiment, the template acquisition unit includes:
对标注框集合对应的至少一个标注框分别进行质量评价，获得至少一个标注框分别对应的质量评价数据；Perform quality evaluation on at least one annotation box corresponding to the annotation box set, and obtain quality evaluation data corresponding to at least one annotation box respectively;
质量选择模块，用于根据至少一个标注框分别对应的质量评价数据，选择质量评价数据最大的标注框为标注框集合的模板标注框。The quality selection module is used to select the annotation box with the largest quality evaluation data as the template annotation box of the annotation box set based on the quality evaluation data corresponding to at least one annotation box.
作为又一个实施例，质量选择模块，包括：As yet another embodiment, the quality selection module includes:
数量确定子模块，用于根据标注框集合对应的至少一个标注框，确定至少一个标注框在对应训练数据包围的点云数据的点云数量；The quantity determination submodule is used to determine the number of point clouds of the point cloud data surrounded by the corresponding training data by at least one annotation frame based on at least one annotation frame corresponding to the annotation frame set;
质量筛选子模块，用于将标注框的在对应训练数据的点云数量作为标注框的质量评价数据，获得至少一个标注框分别对应的质量评价数据。The quality screening submodule is used to use the number of point clouds in the corresponding training data of the annotation frame as the quality evaluation data of the annotation frame, and obtain the quality evaluation data corresponding to at least one annotation frame.
作为又一个实施例，标注分组模块，包括：As another embodiment, the annotation grouping module includes:
向量提取子模块，用于基于训练集中的多个标注框，提取标注框的特征向量，获得多个标注框分别对应的特征向量；The vector extraction submodule is used to extract the feature vectors of the annotation boxes based on multiple annotation boxes in the training set, and obtain the feature vectors corresponding to the multiple annotation boxes;
标注聚类子模块，用于根据聚类算法，将多个标注框分别对应的特征向量进行聚类，获得至少一个标注框集合。The annotation clustering submodule is used to cluster the feature vectors corresponding to multiple annotation boxes according to the clustering algorithm to obtain at least one annotation box set.
作为又一个实施例，还包括：As yet another embodiment, it also includes:
生成单元，用于根据3D点云和目标检测框，生成目标对象的检测结果；The generation unit is used to generate the detection results of the target object based on the 3D point cloud and the target detection frame;
输出单元，用于输出检测结果。Output unit, used to output detection results.
需要说明的是，本实施例中的目标对象并不是针对某一特定用户，并不能反映出某一特定用户的个人信息。需要说明的是，本实施例中的训练集来自于公开数据集。It should be noted that the target object in this embodiment is not aimed at a specific user and cannot reflect the personal information of a specific user. It should be noted that the training set in this embodiment comes from a public data set.
本公开的技术方案中，所涉及的用户个人信息的收集、存储、使用、加工、传输、提供和公开等处理，均符合相关法律法规的规定，且不违背公序良俗。In the technical solution of this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are in compliance with relevant laws and regulations and do not violate public order and good customs.
根据本公开的实施例，本公开还提供了一种电子设备、一种可读存储介质和一种计算机程序产品。According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
根据本公开的实施例，本公开还提供了一种计算机程序产品，计算机程序产品包括：计算机程序，计算机程序存储在可读存储介质中，电子设备的至少一个处理器可以从可读存储介质读取计算机程序，至少一个处理器执行计算机程序使得电子设备执行上述任一实施例提供的方案。According to an embodiment of the present disclosure, the present disclosure also provides a computer program product. The computer program product includes: a computer program. The computer program is stored in a readable storage medium. At least one processor of the electronic device can read from the readable storage medium. Taking a computer program, at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.
图8示出了可以用来实施本公开的实施例的示例电子设备800的示意性框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。Figure 8 shows a schematic block diagram of an example electronic device 800 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit implementations of the disclosure described and/or claimed herein.
如图8所示，设备800包括计算单元801，其可以根据存储在只读存储器(ROM)802中的计算机程序或者从存储单元808加载到随机访问存储器(RAM)803中的计算机程序，来执行各种适当的动作和处理。在RAM 803中，还可存储设备800操作所需的各种程序和数据。计算单元801、ROM 802以及RAM 803通过总线804彼此相连。输入/输出(I/O)接口805也连接至总线804。As shown in FIG. 8 , the device 800 includes a computing unit 801 that can execute according to a computer program stored in a read-only memory (ROM) 802 or loaded from a storage unit 808 into a random access memory (RAM) 803 Various appropriate actions and treatments. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. Computing unit 801, ROM 802 and RAM 803 are connected to each other via bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
设备800中的多个部件连接至I/O接口805，包括：输入单元806，例如键盘、鼠标等；输出单元807，例如各种类型的显示器、扬声器等；存储单元808，例如磁盘、光盘等；以及通信单元809，例如网卡、调制解调器、无线通信收发机等。通信单元809允许设备800通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。Multiple components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc. ; and communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices through computer networks such as the Internet and/or various telecommunications networks.
计算单元801可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元801的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元801执行上文所描述的各个方法和处理，例如基于点云的目标检测方法。例如，在一些实施例中，基于点云的目标检测方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元808。在一些实施例中，计算机程序的部分或者全部可以经由ROM 802和/或通信单元809而被载入和/或安装到设备800上。当计算机程序加载到RAM 803并由计算单元801执行时，可以执行上文描述的基于点云的目标检测方法的一个或多个步骤。备选地，在其他实施例中，计算单元801可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行基于点云的目标检测方法。Computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processing processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs various methods and processes described above, such as a point cloud-based target detection method. For example, in some embodiments, the point cloud-based object detection method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 800 via ROM 802 and/or communication unit 809 . When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the point cloud-based target detection method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the point cloud-based object detection method in any other suitable manner (eg, by means of firmware).
本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、复杂可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described above may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip implemented in a system (SOC), complex programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor The processor, which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions specified in the flowcharts and/or block diagrams/ The operation is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以解决了传统物理主机与VPS服务("Virtual Private Server"，或简称"VPS")中，存在的管理难度大，业务扩展性弱的缺陷。服务器也可以为分布式系统的服务器，或者是结合了区块链的服务器。Computer systems may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. The relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship with each other. The server can be a cloud server, also known as cloud computing server or cloud host. It is a host product in the cloud computing service system to solve the problem of traditional physical host and VPS service ("Virtual Private Server", or "VPS" for short) Among them, there are defects such as difficult management and weak business scalability. The server can also be a distributed system server or a server combined with a blockchain.
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本发公开中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。It should be understood that various forms of the process shown above may be used, with steps reordered, added or deleted. For example, each step described in the present disclosure can be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solution disclosed in the present disclosure can be achieved, there is no limitation here.
上述具体实施方式，并不构成对本公开保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本公开的精神和原则之内所作的修改、等同替换和改进等，均应包含在本公开保护范围之内。The above-mentioned specific embodiments do not constitute a limitation on the scope of the present disclosure. It will be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this disclosure shall be included in the protection scope of this disclosure.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311209920.XACN117237634A (en) | 2023-09-18 | 2023-09-18 | Point cloud-based target detection methods, devices, equipment, media and products |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311209920.XACN117237634A (en) | 2023-09-18 | 2023-09-18 | Point cloud-based target detection methods, devices, equipment, media and products |
| Publication Number | Publication Date |
|---|---|
| CN117237634Atrue CN117237634A (en) | 2023-12-15 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311209920.XAPendingCN117237634A (en) | 2023-09-18 | 2023-09-18 | Point cloud-based target detection methods, devices, equipment, media and products |
| Country | Link |
|---|---|
| CN (1) | CN117237634A (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109948497A (en)* | 2019-03-12 | 2019-06-28 | 北京旷视科技有限公司 | A kind of object detecting method, device and electronic equipment |
| CN112505652A (en)* | 2021-02-04 | 2021-03-16 | 知行汽车科技（苏州）有限公司 | Target detection method, device and storage medium |
| US20220067532A1 (en)* | 2020-08-31 | 2022-03-03 | Si Analytics Co., Ltd | Method to Train Model |
| CN115035359A (en)* | 2021-02-24 | 2022-09-09 | 华为技术有限公司 | A point cloud data processing method, training data processing method and device |
| WO2022213879A1 (en)* | 2021-04-07 | 2022-10-13 | 腾讯科技（深圳）有限公司 | Target object detection method and apparatus, and computer device and storage medium |
| CN115830571A (en)* | 2022-10-31 | 2023-03-21 | 惠州市德赛西威智能交通技术研究院有限公司 | Method, device and equipment for determining detection frame and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109948497A (en)* | 2019-03-12 | 2019-06-28 | 北京旷视科技有限公司 | A kind of object detecting method, device and electronic equipment |
| US20220067532A1 (en)* | 2020-08-31 | 2022-03-03 | Si Analytics Co., Ltd | Method to Train Model |
| CN112505652A (en)* | 2021-02-04 | 2021-03-16 | 知行汽车科技（苏州）有限公司 | Target detection method, device and storage medium |
| CN115035359A (en)* | 2021-02-24 | 2022-09-09 | 华为技术有限公司 | A point cloud data processing method, training data processing method and device |
| WO2022213879A1 (en)* | 2021-04-07 | 2022-10-13 | 腾讯科技（深圳）有限公司 | Target object detection method and apparatus, and computer device and storage medium |
| CN115830571A (en)* | 2022-10-31 | 2023-03-21 | 惠州市德赛西威智能交通技术研究院有限公司 | Method, device and equipment for determining detection frame and storage medium |
| Publication | Publication Date | Title |
|---|---|---|
| CN113012210B (en) | Method and device for generating depth map, electronic equipment and storage medium | |
| CN113989450B (en) | Image processing method, device, electronic equipment and medium | |
| US12169942B2 (en) | Method for training image depth estimation model and method for processing image depth information | |
| CN114279433B (en) | Automatic map data production method, related device and computer program product | |
| CN113657274B (en) | Table generation method and device, electronic equipment and storage medium | |
| WO2022183500A1 (en) | Projection-based point cloud quality evaluation method and apparatus, device and storage medium | |
| WO2021155558A1 (en) | Road marking identification method, map generation method and related product | |
| CN114565916B (en) | Target detection model training method, target detection method and electronic equipment | |
| CN113762397B (en) | Method, equipment, medium and product for training detection model and updating high-precision map | |
| CN112509126B (en) | Method, device, equipment and storage medium for detecting three-dimensional object | |
| CN115410173B (en) | High-precision map element recognition method, device, equipment and medium based on multi-modal fusion | |
| CN114612544B (en) | Image processing method, device, equipment and storage medium | |
| CN115880536B (en) | Data processing method, training method, target object detection method and device | |
| WO2021189420A1 (en) | Data processing method and device | |
| CN115018753A (en) | Projection-based point cloud quality evaluation method, device, equipment and storage medium | |
| CN114611635B (en) | Object recognition method, device, storage medium and electronic device | |
| CN116129069A (en) | Calculation method, device, electronic equipment and storage medium of planar area | |
| CN115937449A (en) | High-precision map generation method and device, electronic equipment and storage medium | |
| CN114529801A (en) | Target detection method, device, equipment and storage medium | |
| CN116740498B (en) | Model pre-training method, model training method, object processing method and device | |
| CN117746134A (en) | Tag generation method, device and equipment of detection frame and storage medium | |
| CN117237634A (en) | Point cloud-based target detection methods, devices, equipment, media and products | |
| CN117593749A (en) | Point cloud panoramic segmentation, data processing methods, devices, equipment, media and products | |
| CN117611643A (en) | Point cloud registration method and device and electronic equipment | |
| CN115578337A (en) | A point cloud detection and optimization method, device, electronic equipment and storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |