技术领域Technical Field
本申请实施例涉及计算机技术领域，具体涉及用于输出信息的方法和装置。The embodiments of the present application relate to the field of computer technology, and more particularly to a method and device for outputting information.
背景技术Background technique
末端物流配送机器人可以极大地降低物料配送成本，提高配送效率。但由于车体较小，并且出于成本的考虑，末端物流配送机器人的车载计算单元的计算能力有限。这样，末端物流配送机器人不能运行较复杂的算法来识别其行驶环境中的障碍物。提供耗时较小，不需要较高的计算资源的聚类算法，对末端物流配送机器人来说至关重要。Terminal logistics delivery robots can greatly reduce material delivery costs and improve delivery efficiency. However, due to the small size of the vehicle and cost considerations, the computing power of the onboard computing unit of the terminal logistics delivery robot is limited. As a result, the terminal logistics delivery robot cannot run more complex algorithms to identify obstacles in its driving environment. Providing a clustering algorithm that takes less time and does not require high computing resources is crucial for terminal logistics delivery robots.
发明内容Summary of the invention
本申请实施例提出了用于输出信息的方法和装置。The embodiments of the present application provide a method and device for outputting information.
第一方面，本申请实施例提供了一种用于输出信息的方法，包括：获取车辆在行驶过程中采集的图像数据和点云数据；识别上述图像数据中的障碍物，以及采用标注框标注上述障碍物；根据上述点云数据，确定上述标注框中各像素点对应的点云点的位置信息；根据各点云点的位置信息，确定上述障碍物的中心位置；将上述中心位置输出。In a first aspect, an embodiment of the present application provides a method for outputting information, including: acquiring image data and point cloud data collected by a vehicle during driving; identifying obstacles in the above image data, and marking the above obstacles with a labeling box; determining, based on the above point cloud data, position information of point cloud points corresponding to each pixel point in the above labeling box; determining the center position of the above obstacle based on the position information of each point cloud point; and outputting the above center position.
在一些实施例中，上述根据各点云点的位置信息，确定上述障碍物的中心位置，包括：根据各点云点的位置信息，确定位于预设的不同位置区间的点云点的数量；根据各位置区间中包括的点云点的数量，从各位置区间中确定出目标位置区间；根据上述目标位置区间中包括的点云点，确定上述障碍物的中心位置。In some embodiments, the above-mentioned determining the center position of the above-mentioned obstacle based on the position information of each point cloud point includes: determining the number of point cloud points located in preset different position intervals based on the position information of each point cloud point; determining the target position interval from each position interval based on the number of point cloud points included in each position interval; and determining the center position of the above-mentioned obstacle based on the point cloud points included in the above-mentioned target position interval.
在一些实施例中，上述根据上述目标位置区间中包括的点云点，确定上述障碍物的中心位置，包括：确定包络上述目标位置区间中各点云点的包络框；将上述包络框的中心点的位置作为上述障碍物的中心位置。In some embodiments, determining the center position of the obstacle based on the point cloud points included in the target position interval includes: determining an envelope box that envelops each point cloud point in the target position interval; and taking the position of the center point of the envelope box as the center position of the obstacle.
在一些实施例中，上述方法还包括：根据上述障碍物的中心位置，对上述障碍物进行轨迹追踪。In some embodiments, the method further includes: tracking the trajectory of the obstacle according to the center position of the obstacle.
在一些实施例中，上述方法还包括：根据上述障碍物的中心位置，确定上述障碍物的速度。In some embodiments, the method further includes: determining the speed of the obstacle based on the center position of the obstacle.
第二方面，本申请实施例提供了一种用于输出信息的装置，包括：获取单元，被配置成获取车辆在行驶过程中采集的图像数据和点云数据；识别单元，被配置成识别上述图像数据中的障碍物，以及采用标注框标注上述障碍物；划分单元，被配置成根据上述点云数据，确定上述标注框中各像素点对应的点云点的位置信息；确定单元，被配置成根据各点云点的位置信息，确定上述障碍物的中心位置；输出单元，被配置成将上述中心位置输出。In a second aspect, an embodiment of the present application provides a device for outputting information, comprising: an acquisition unit, configured to acquire image data and point cloud data collected by a vehicle during driving; an identification unit, configured to identify obstacles in the above image data, and to mark the above obstacles with a marking box; a division unit, configured to determine, based on the above point cloud data, position information of point cloud points corresponding to each pixel point in the above marking box; a determination unit, configured to determine the center position of the above obstacle based on the position information of each point cloud point; and an output unit, configured to output the above center position.
在一些实施例中，上述确定单元包括：数量确定模块，被配置成根据各点云点的位置信息，确定位于预设的不同位置区间的点云点的数量；区间确定模块，被配置成根据各位置区间中包括的点云点的数量，从各位置区间中确定出目标位置区间；位置确定模块，被配置成根据上述目标位置区间中包括的点云点，确定上述障碍物的中心位置。In some embodiments, the above-mentioned determination unit includes: a quantity determination module, configured to determine the number of point cloud points located in preset different position intervals according to the position information of each point cloud point; an interval determination module, configured to determine the target position interval from each position interval according to the number of point cloud points included in each position interval; and a position determination module, configured to determine the center position of the above-mentioned obstacle according to the point cloud points included in the above-mentioned target position interval.
在一些实施例中，上述位置确定模块进一步被配置成：确定包络上述目标位置区间中各点云点的包络框；将上述包络框的中心点的位置作为上述障碍物的中心位置。In some embodiments, the position determination module is further configured to: determine an envelope frame that encloses each point cloud point in the target position interval; and use the position of the center point of the envelope frame as the center position of the obstacle.
在一些实施例中，上述装置还包括：轨迹追踪单元，被配置成根据上述障碍物的中心位置，对上述障碍物进行轨迹追踪。In some embodiments, the above-mentioned device further includes: a trajectory tracking unit configured to track the trajectory of the above-mentioned obstacle according to the center position of the above-mentioned obstacle.
在一些实施例中，上述装置还包括：速度确定单元，被配置成根据上述障碍物的中心位置，确定上述障碍物的速度。In some embodiments, the apparatus further comprises: a speed determination unit configured to determine the speed of the obstacle according to the center position of the obstacle.
第三方面，本申请实施例提供了一种电子设备，包括：一个或多个处理器；存储装置，其上存储有一个或多个程序，当上述一个或多个程序被上述一个或多个处理器执行，使得上述一个或多个处理器实现如第一方面任一实施例所描述的方法。In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any embodiment of the first aspect.
第四方面，本申请实施例提供了一种计算机可读介质，其上存储有计算机程序，该程序被处理器执行时实现如第一方面任一实施例所描述的方法。In a fourth aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of the first aspect.
本申请的上述实施例提供的用于输出信息的方法和装置，首先可以获取车辆在行驶过程中采集的图像数据和点云数据。然后识别图像数据中的障碍物，并利用标注框标注上述障碍物。并根据点云数据，确定标注框中各像素点对应的位置信息。最后根据各像素点对应的点云点的位置信息，确定障碍物的中心位置。并将中心位置输出。本实施例的方法，可以快速的根据障碍物的各点云点的位置信息，来确定障碍物的中心位置，从而能够在低成本的情况下提高数据的处理效率。The method and device for outputting information provided by the above-mentioned embodiments of the present application can first obtain image data and point cloud data collected by the vehicle during driving. Then, obstacles in the image data are identified, and the above obstacles are marked with a labeling box. And based on the point cloud data, the position information corresponding to each pixel point in the labeling box is determined. Finally, the center position of the obstacle is determined based on the position information of the point cloud points corresponding to each pixel point. And the center position is output. The method of this embodiment can quickly determine the center position of the obstacle based on the position information of each point cloud point of the obstacle, thereby improving the data processing efficiency at a low cost.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本申请的其它特征、目的和优点将会变得更明显：Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
图1是本申请的一个实施例可以应用于其中的示例性系统架构图；FIG1 is an exemplary system architecture diagram in which an embodiment of the present application can be applied;
图2是根据本申请的用于输出信息的方法的一个实施例的流程图；FIG2 is a flow chart of an embodiment of a method for outputting information according to the present application;
图3是根据本申请的用于输出信息的方法的一个应用场景的示意图；FIG3 is a schematic diagram of an application scenario of the method for outputting information according to the present application;
图4是根据本申请的用于输出信息的方法中确定障碍物的中心位置的一个实施例的流程图；FIG4 is a flow chart of an embodiment of determining the central position of an obstacle in a method for outputting information according to the present application;
图5是图4所示实施例的一个应用场景的示意图；FIG5 is a schematic diagram of an application scenario of the embodiment shown in FIG4 ;
图6是根据本申请的用于输出信息的装置的一个实施例的结构示意图；FIG6 is a schematic structural diagram of an embodiment of a device for outputting information according to the present application;
图7是适于用来实现本申请实施例的电子设备的计算机系统的结构示意图。FIG. 7 is a schematic diagram of the structure of a computer system of an electronic device suitable for implementing an embodiment of the present application.
具体实施方式Detailed ways
下面结合附图和实施例对本申请作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释相关发明，而非对该发明的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与有关发明相关的部分。The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。下面将参考附图并结合实施例来详细说明本申请。It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
图1示出了可以应用本申请的用于输出信息的方法或用于输出信息的装置的实施例的示例性系统架构100。FIG. 1 shows an exemplary system architecture 100 to which an embodiment of a method for outputting information or an apparatus for outputting information of the present application can be applied.
如图1所示，系统架构100可以包括车辆101，末端物流配送机器人102、103，网络104和服务器105。网络104用以在车辆101，末端物流配送机器人102、103和服务器105之间提供通信链路的介质。网络104可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。As shown in FIG1 , the system architecture 100 may include a vehicle 101, terminal logistics delivery robots 102 and 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the vehicle 101, the terminal logistics delivery robots 102 and 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.
车辆101以及末端物流配送机器人102、103上可以安装有数据采集装置，例如激光雷达传感器、深度照相机、单目相机或双目相机等，以采集其在行驶过程中行驶环境的信息。车辆101以及末端物流配送机器人102、103可以实时将自身的位置或状态通过网络104发送给服务器105，以接收服务器105发送的信息。The vehicle 101 and the terminal logistics delivery robots 102 and 103 may be equipped with data acquisition devices, such as laser radar sensors, depth cameras, monocular cameras or binocular cameras, etc., to collect information about the driving environment during driving. The vehicle 101 and the terminal logistics delivery robots 102 and 103 may send their own positions or states to the server 105 via the network 104 in real time to receive the information sent by the server 105.
需要说明的是，本申请实施例所提供的用于输出信息的方法一般由车辆101或末端物流配送机器人102、103执行。相应地，用于输出信息的装置一般设置于车辆101或末端物流配送机器人102、103中。It should be noted that the method for outputting information provided in the embodiment of the present application is generally performed by the vehicle 101 or the terminal logistics delivery robot 102, 103. Accordingly, the device for outputting information is generally set in the vehicle 101 or the terminal logistics delivery robot 102, 103.
应该理解，图1中的车辆、末端物流配送机器人、网络和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目的车辆、末端物流配送机器人、网络和服务器。It should be understood that the number of vehicles, terminal logistics delivery robots, networks and servers in Figure 1 is only illustrative. According to the implementation requirements, there can be any number of vehicles, terminal logistics delivery robots, networks and servers.
继续参考图2，示出了根据本申请的用于输出信息的方法的一个实施例的流程200。本实施例的用于输出信息的方法，包括以下步骤：Continuing to refer to FIG2 , a process 200 of an embodiment of a method for outputting information according to the present application is shown. The method for outputting information in this embodiment includes the following steps:
步骤201，获取车辆在行驶过程中采集的图像数据和点云数据。Step 201, acquiring image data and point cloud data collected during the driving process of the vehicle.
在本实施例中，用于输出信息的方法的执行主体(例如图1所示的车辆101或末端物流配送机器人102、103)可以通过有线连接方式或者无线连接方式获取车辆在行驶过程中采集的图像数据和点云数据。执行主体上可以安装有图像采集装置和点云采集装置。图像采集装置可以包括单目相机、双目相机等。点云采集装置可以包括激光雷达传感器、深度相机等。可以理解的是，图像数据和点云数据中包括行驶环境的信息、障碍物的信息。并且，图像数据中的像素点与点云数据中的点云点是一一对应的，即图像采集装置和点云采集装置是预先标定好的。In this embodiment, the execution entity of the method for outputting information (such as the vehicle 101 or the terminal logistics delivery robots 102, 103 shown in Figure 1) can obtain the image data and point cloud data collected by the vehicle during driving through a wired connection or a wireless connection. An image acquisition device and a point cloud acquisition device can be installed on the execution entity. The image acquisition device may include a monocular camera, a binocular camera, etc. The point cloud acquisition device may include a lidar sensor, a depth camera, etc. It can be understood that the image data and the point cloud data include information about the driving environment and information about obstacles. In addition, the pixel points in the image data correspond one-to-one to the point cloud points in the point cloud data, that is, the image acquisition device and the point cloud acquisition device are pre-calibrated.
步骤202，识别图像数据中的障碍物，以及采用标注框标注障碍物。Step 202: identifying obstacles in the image data and marking the obstacles with a marking box.
在获取到图像数据后，执行主体可以识别图像数据中的障碍物。具体的，执行主体可以采用障碍物识别算法或者预先训练的障碍物识别模型来识别图像数据中的障碍物。在识别出图像数据中包括的障碍物后，执行主体可以采用标注框标注识别出的障碍物。上述标注框可以是各种类型的标注框，例如矩形、椭圆形等等。在一些应用场景中，执行主体可以采用不同类型的标注框来标注不同类型的障碍物。After acquiring the image data, the execution subject may identify obstacles in the image data. Specifically, the execution subject may use an obstacle recognition algorithm or a pre-trained obstacle recognition model to identify obstacles in the image data. After identifying the obstacles included in the image data, the execution subject may use a labeling box to mark the identified obstacles. The above-mentioned labeling box may be various types of labeling boxes, such as a rectangle, an ellipse, etc. In some application scenarios, the execution subject may use different types of labeling boxes to mark different types of obstacles.
步骤203，根据点云数据，确定标注框中各像素点对应的点云点的位置信息。Step 203, determining the position information of the point cloud points corresponding to each pixel point in the annotation box according to the point cloud data.
本实施例中，点云数据可以包括激光点云数据和深度点云数据。激光点云数据中包括了各点云点的坐标和强度，深度点云数据中包括了各点云点与执行主体之间的距离。执行主体可以根据上述点云数据，来确定标注框中的各像素点所对应的点云点的位置信息。In this embodiment, the point cloud data may include laser point cloud data and depth point cloud data. The laser point cloud data includes the coordinates and intensity of each point cloud point, and the depth point cloud data includes the distance between each point cloud point and the execution subject. The execution subject can determine the position information of the point cloud point corresponding to each pixel point in the annotation box based on the above point cloud data.
步骤204，根据各点云点的位置信息，确定障碍物的中心位置。Step 204: Determine the center position of the obstacle based on the position information of each point cloud point.
执行主体在确定了标注框中各点云点的位置信息后，可以根据各位置信息，确定障碍物的中心位置。例如，执行主体可以计算各位置信息的平均值，并确定位置信息等于平均值的各点云点的位置。然后，将各点云点的中心位置作为障碍物的中心位置。或者，执行主体可以将位置信息在预设位置区间的点云点的中心位置作为障碍物的中心位置。After determining the position information of each point cloud point in the annotation box, the execution entity can determine the center position of the obstacle based on each position information. For example, the execution entity can calculate the average value of each position information and determine the position of each point cloud point whose position information is equal to the average value. Then, the center position of each point cloud point is used as the center position of the obstacle. Alternatively, the execution entity can use the center position of the point cloud point whose position information is in a preset position interval as the center position of the obstacle.
步骤205，将中心位置输出。Step 205, output the center position.
执行主体在确定障碍物的中心位置后，可以将中心位置输出，以进行后续的处理。例如，对障碍物的速度进行预测、对障碍物的轨迹进行追踪，对执行主体本身的行驶策略进行规划等等。After determining the center position of the obstacle, the execution subject can output the center position for subsequent processing, such as predicting the speed of the obstacle, tracking the trajectory of the obstacle, and planning the driving strategy of the execution subject itself.
在本实施例的一些可选的实现方式中，上述方法还可以进一步包括图2中未示出的以下步骤：根据障碍物的中心位置，对障碍物进行轨迹追踪。In some optional implementations of this embodiment, the above method may further include the following steps not shown in FIG. 2 : tracking the trajectory of the obstacle according to the center position of the obstacle.
本实现方式中，执行主体可以实时获取车辆行驶过程中的图像数据和点云数据，这样可以实时确定出障碍物的中心位置。执行主体可以根据各时刻障碍物的中心位置，来对障碍物进行轨迹追踪。In this implementation, the execution subject can obtain the image data and point cloud data of the vehicle in real time, so that the center position of the obstacle can be determined in real time. The execution subject can track the obstacle according to the center position of the obstacle at each moment.
在本实施例的一些可选的实现方式中，上述方法还可以进一步包括图2中未示出的以下步骤：根据障碍物的中心位置，确定障碍物的速度。In some optional implementations of this embodiment, the above method may further include the following steps not shown in FIG. 2 : determining the speed of the obstacle according to the center position of the obstacle.
本实现方式中，执行主体还可以根据相邻时刻障碍物的中心位置，来计算障碍物的行驶速度。得到的行驶速度可以用于指导执行主体自身的行驶速度和方向。In this implementation, the execution subject can also calculate the speed of the obstacle according to the center position of the obstacle at adjacent moments. The obtained speed can be used to guide the execution subject's own speed and direction.
继续参见图3，图3是根据本实施例的用于输出信息的方法的一个应用场景的示意图。在图3的应用场景中，末端物料配送机器人内装有快递，其在配送过程中可以通过机器人上安装的双目相机来采集行驶环境的图像数据和点云数据，并根据步骤202～204的处理，确定出行驶环境中的障碍物的中心位置。然后根据上述中心位置来确定自身的行驶速度和行驶方向。Continuing to refer to FIG. 3, FIG. 3 is a schematic diagram of an application scenario of the method for outputting information according to this embodiment. In the application scenario of FIG. 3, the terminal material delivery robot is equipped with express delivery. During the delivery process, the robot can collect image data and point cloud data of the driving environment through the binocular camera installed on the robot, and determine the center position of obstacles in the driving environment according to the processing of steps 202 to 204. Then, the driving speed and driving direction of the robot are determined according to the above center position.
本申请的上述实施例提供的用于输出信息的方法，首先可以获取车辆在行驶过程中采集的图像数据和点云数据。然后识别图像数据中的障碍物，并利用标注框标注上述障碍物。并根据点云数据，确定标注框中各像素点对应的位置信息。最后根据各像素点对应的点云点的位置信息，确定障碍物的中心位置。并将中心位置输出。本实施例的方法，可以快速的根据障碍物的各像素点对应的位置信息，来确定障碍物的中心位置，从而能够在低成本的情况下提高数据的处理效率。The method for outputting information provided by the above-mentioned embodiment of the present application can first obtain image data and point cloud data collected by the vehicle during driving. Then, obstacles in the image data are identified, and the above obstacles are marked with a labeling box. And based on the point cloud data, the position information corresponding to each pixel point in the labeling box is determined. Finally, the center position of the obstacle is determined based on the position information of the point cloud point corresponding to each pixel point. And the center position is output. The method of this embodiment can quickly determine the center position of the obstacle based on the position information corresponding to each pixel point of the obstacle, thereby improving the data processing efficiency at a low cost.
继续参见图4，其示出了根据本申请的用于输出信息的方法中确定障碍物的中心位置的一个实施例的流程400。如图4所示，本实施例的用于输出信息的方法，可以通过以下步骤来确定障碍物的中心位置：Continuing to refer to FIG4, it shows a process 400 of determining the center position of an obstacle in a method for outputting information according to an embodiment of the present application. As shown in FIG4, the method for outputting information of this embodiment can determine the center position of an obstacle by the following steps:
步骤401，根据各点云点的位置信息，确定位于预设的不同位置区间的像素点的数量。Step 401, determining the number of pixel points located in different preset position intervals according to the position information of each point cloud point.
执行主体可以根据各点云点的位置信息，对各点云点进行划分。具体的，执行主体可以将各点云点根据位置信息划分到不同的位置区间，从而可以统计出位于各位置区间的点云点的数量。例如，执行主体可以统计出x坐标和y坐标分别位于0～d、d～2d、2d～3d……之间的各点云点的数量。The execution subject may divide each point cloud point according to the position information of each point cloud point. Specifically, the execution subject may divide each point cloud point into different position intervals according to the position information, so as to count the number of point cloud points located in each position interval. For example, the execution subject may count the number of each point cloud point whose x-coordinate and y-coordinate are respectively located between 0 and d, d and 2d, 2d and 3d, etc.
步骤402，根据各位置区间中包括的点云点的数量，从各位置区间中确定出目标位置区间。Step 402: determining a target position interval from each position interval according to the number of point cloud points included in each position interval.
在确定各位置区间中包括的点云点的数量后，执行主体可以从各个位置区间中确定出目标位置区间。执行主体可以将包括点云点数量最多的位置区间作为目标位置区间。或者，执行主体可以将包括点云点数量前N(N为大于1的自然数)的位置区间作为目标位置区间。执行主体可以根据目标位置区间中包括的点云点，确定障碍物的中心位置。After determining the number of point cloud points included in each position interval, the execution subject can determine the target position interval from each position interval. The execution subject can use the position interval with the largest number of point cloud points as the target position interval. Alternatively, the execution subject can use the position interval including the first N (N is a natural number greater than 1) of the number of point cloud points as the target position interval. The execution subject can determine the center position of the obstacle based on the point cloud points included in the target position interval.
步骤403，确定包络目标位置区间中各点云点的包络框。Step 403: determine the envelope of each point cloud point in the envelope target position interval.
执行主体可以确定能够包络目标位置区间中各点云点的包络框。可以理解的是，该包络框是一个三维的立体框。具体的，执行主体可以计算目标位置区间中各点云点的最小外接球体，并将该最小外接球体作为各点云点的包络框。The execution subject can determine an envelope that can envelop each point cloud point in the target position interval. It can be understood that the envelope is a three-dimensional solid frame. Specifically, the execution subject can calculate the minimum circumscribed sphere of each point cloud point in the target position interval, and use the minimum circumscribed sphere as the envelope of each point cloud point.
步骤404，将包络框的中心点的位置作为障碍物的中心位置。Step 404: The position of the center point of the envelope is used as the center position of the obstacle.
执行主体可以计算包络框的中心点，然后将上述中心点的位置作为障碍物的中心位置。The execution subject may calculate the center point of the envelope, and then use the position of the center point as the center position of the obstacle.
继续参见图5，其示出了图4所示实施例的一个应用场景的示意图。在图5中，障碍物为末端物料配送机器人前方的行人，末端物料配送机器人通过双目相机采集到行人的图像数据和点云数据，并对图像中的行人标注了标注框。然后对点云数据进行划分，得到深度统计直方图，其表示了位于各距离区间的点云点的数量。然后根据包括点云点数量最多的区间包括的点云点来计算行人的中心位置。Continuing to refer to FIG5, it shows a schematic diagram of an application scenario of the embodiment shown in FIG4. In FIG5, the obstacle is a pedestrian in front of the terminal material delivery robot. The terminal material delivery robot collects the image data and point cloud data of the pedestrian through a binocular camera, and marks the pedestrian in the image with a label box. Then the point cloud data is divided to obtain a depth statistical histogram, which represents the number of point cloud points located in each distance interval. Then the center position of the pedestrian is calculated based on the point cloud points included in the interval with the largest number of point cloud points.
进一步参考图6，作为对上述各图所示方法的实现，本申请提供了一种用于输出信息的装置的一个实施例，该装置实施例与图2所示的方法实施例相对应，该装置具体可以应用于各种电子设备中。Further referring to FIG. 6 , as an implementation of the methods shown in the above figures, the present application provides an embodiment of a device for outputting information. The device embodiment corresponds to the method embodiment shown in FIG. 2 , and the device can be specifically applied to various electronic devices.
如图6所示，本实施例的用于输出信息的装置600包括：获取单元601、识别单元602、划分单元603、确定单元604和输出单元605。As shown in FIG. 6 , the apparatus 600 for outputting information in this embodiment includes: an acquiring unit 601 , an identifying unit 602 , a dividing unit 603 , a determining unit 604 , and an output unit 605 .
获取单元601，被配置成获取车辆在行驶过程中采集的图像数据和点云数据。The acquisition unit 601 is configured to acquire image data and point cloud data collected during the driving process of the vehicle.
识别单元602，被配置成识别图像数据中的障碍物，以及采用标注框标注障碍物。The recognition unit 602 is configured to recognize obstacles in the image data and mark the obstacles with a marking box.
划分单元603，被配置成根据点云数据，确定标注框中各像素点对应的点云点的位置信息。The dividing unit 603 is configured to determine the position information of the point cloud points corresponding to each pixel point in the annotation box according to the point cloud data.
确定单元604，被配置成根据各点云点的位置信息，确定障碍物的中心位置。The determination unit 604 is configured to determine the center position of the obstacle according to the position information of each point cloud point.
输出单元605，被配置成将中心位置输出。The output unit 605 is configured to output the center position.
在本实施例的一些可选的实现方式中，确定单元604可以进一步包括图6中未示出的：数量确定模块、区间确定模块和位置确定模块。In some optional implementations of this embodiment, the determining unit 604 may further include: a quantity determining module, an interval determining module, and a position determining module, which are not shown in FIG. 6 .
数量确定模块，被配置成根据各点云点的位置信息，确定位于预设的不同位置区间的点云点的数量。The quantity determination module is configured to determine the quantity of point cloud points located in different preset position intervals according to the position information of each point cloud point.
区间确定模块，被配置成根据各位置区间中包括的点云点的数量，从各位置区间中确定出目标位置区间。The interval determination module is configured to determine a target position interval from each position interval according to the number of point cloud points included in each position interval.
位置确定模块，被配置成根据目标位置区间中包括的点云点，确定障碍物的中心位置。The position determination module is configured to determine the center position of the obstacle based on the point cloud points included in the target position interval.
在本实施例的一些可选的实现方式中，位置确定模块可以进一步被配置成：确定包络目标位置区间中各点云点的包络框；将包络框的中心点的位置作为障碍物的中心位置。In some optional implementations of this embodiment, the position determination module may be further configured to: determine an envelope frame of each point cloud point in the envelope target position interval; and use the position of the center point of the envelope frame as the center position of the obstacle.
在本实施例的一些可选的实现方式中，装置600还可以进一步包括图6中未示出的轨迹追踪单元，被配置成根据障碍物的中心位置，对障碍物进行轨迹追踪。In some optional implementations of this embodiment, the device 600 may further include a trajectory tracking unit not shown in FIG. 6 , which is configured to track the trajectory of the obstacle according to the center position of the obstacle.
在本实施例的一些可选的实现方式中，装置600还可以进一步包括图6中未示出的速度确定单元，被配置成根据障碍物的中心位置，确定障碍物的速度。In some optional implementations of this embodiment, the device 600 may further include a speed determination unit not shown in FIG. 6 , which is configured to determine the speed of the obstacle according to the center position of the obstacle.
应当理解，用于输出信息的装置600中记载的单元601至单元605分别与参考图2中描述的方法中的各个步骤相对应。由此，上文针对用于输出信息的方法描述的操作和特征同样适用于装置600及其中包含的单元，在此不再赘述。It should be understood that the units 601 to 605 recorded in the device 600 for outputting information respectively correspond to the steps in the method described with reference to Figure 2. Therefore, the operations and features described above for the method for outputting information are also applicable to the device 600 and the units contained therein, and will not be repeated here.
下面参考图7，其示出了适于用来实现本公开的实施例的电子设备700的结构示意图。图7示出的电子设备仅仅是一个示例，不应对本公开的实施例的功能和使用范围带来任何限制。Referring to Fig. 7, a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiments of the present disclosure is shown. The electronic device shown in Fig. 7 is only an example and should not bring any limitation to the functions and application scope of the embodiments of the present disclosure.
如图7所示，电子设备700可以包括处理装置(例如中央处理器、图形处理器等)701，其可以根据存储在只读存储器(ROM)702中的程序或者从存储装置708加载到随机访问存储器(RAM)703中的程序而执行各种适当的动作和处理。在RAM 703中，还存储有电子设备700操作所需的各种程序和数据。处理装置701、ROM 702以及RAM703通过总线704彼此相连。输入/输出(I/O)接口705也连接至总线704。As shown in FIG7 , the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input/output (I/O) interface 705 is also connected to the bus 704.
通常，以下装置可以连接至I/O接口705：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置706；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置707；包括例如磁带、硬盘等的存储装置708；以及通信装置709。通信装置709可以允许电子设备700与其他设备进行无线或有线通信以交换数据。虽然图7示出了具有各种装置的电子设备700，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。图7中示出的每个方框可以代表一个装置，也可以根据需要代表多个装置。Typically, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication device 709 may allow the electronic device 700 to communicate wirelessly or wired with other devices to exchange data. Although FIG. 7 shows an electronic device 700 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively. Each box shown in FIG. 7 may represent one device, or may represent multiple devices as needed.
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置709从网络上被下载和安装，或者从存储装置708被安装，或者从ROM 702被安装。在该计算机程序被处理装置701执行时，执行本公开的实施例的方法中限定的上述功能。需要说明的是，本公开的实施例所述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开的实施例中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开的实施例中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above functions defined in the method of the embodiment of the present disclosure are executed. It should be noted that the computer-readable medium described in the embodiment of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus, or a device. In an embodiment of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus, or a device. The program code contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：获取车辆在行驶过程中采集的图像数据和点云数据；识别图像数据中的障碍物，以及采用标注框标注障碍物；根据点云数据，确定标注框中各像素点对应的点云点的位置信息；根据各点云点的位置信息，确定障碍物的中心位置；将中心位置输出。The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains image data and point cloud data collected by the vehicle during driving; identifies obstacles in the image data, and annotates obstacles with annotation boxes; determines the position information of the point cloud points corresponding to each pixel point in the annotation box according to the point cloud data; determines the center position of the obstacle according to the position information of each point cloud point; and outputs the center position.
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的实施例的操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
描述于本公开的实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。所描述的单元也可以设置在处理器中，例如，可以描述为：一种处理器包括获取单元、识别单元、划分单元、确定单元和输出单元。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定，例如，获取单元还可以被描述为“获取车辆在行驶过程中采集的图像数据和点云数据的单元”。The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The units described may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, an identification unit, a division unit, a determination unit, and an output unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring image data and point cloud data collected by the vehicle during driving".
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开的实施例中所涉及的发明范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述发明构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开的实施例中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) to form a technical solution.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910983046.2ACN112668371B (en) | 2019-10-16 | 2019-10-16 | Method and apparatus for outputting information |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910983046.2ACN112668371B (en) | 2019-10-16 | 2019-10-16 | Method and apparatus for outputting information |
| Publication Number | Publication Date |
|---|---|
| CN112668371A CN112668371A (en) | 2021-04-16 |
| CN112668371Btrue CN112668371B (en) | 2024-04-09 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910983046.2AActiveCN112668371B (en) | 2019-10-16 | 2019-10-16 | Method and apparatus for outputting information |
| Country | Link |
|---|---|
| CN (1) | CN112668371B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113126120B (en)* | 2021-04-25 | 2023-08-25 | 北京百度网讯科技有限公司 | Data labeling method, device, equipment, storage medium and computer program product |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013065120A1 (en)* | 2011-11-01 | 2013-05-10 | アイシン精機株式会社 | Obstacle alert device |
| CN108256577A (en)* | 2018-01-18 | 2018-07-06 | 东南大学 | A kind of barrier clustering method based on multi-line laser radar |
| CN108985171A (en)* | 2018-06-15 | 2018-12-11 | 上海仙途智能科技有限公司 | Estimation method of motion state and state estimation device |
| CN109344804A (en)* | 2018-10-30 | 2019-02-15 | 百度在线网络技术（北京）有限公司 | A kind of recognition methods of laser point cloud data, device, equipment and medium |
| CN109360239A (en)* | 2018-10-24 | 2019-02-19 | 长沙智能驾驶研究院有限公司 | Obstacle detection method, device, computer equipment and storage medium |
| CN109840448A (en)* | 2017-11-24 | 2019-06-04 | 百度在线网络技术（北京）有限公司 | Information output method and device for automatic driving vehicle |
| CN110068814A (en)* | 2019-03-27 | 2019-07-30 | 东软睿驰汽车技术（沈阳）有限公司 | A kind of method and device measuring obstacle distance |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145680B (en)* | 2017-06-16 | 2022-05-27 | 阿波罗智能技术(北京)有限公司 | A method, apparatus, device and computer storage medium for obtaining obstacle information |
| JP6911777B2 (en)* | 2018-01-23 | 2021-07-28 | トヨタ自動車株式会社 | Motion trajectory generator |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013065120A1 (en)* | 2011-11-01 | 2013-05-10 | アイシン精機株式会社 | Obstacle alert device |
| CN109840448A (en)* | 2017-11-24 | 2019-06-04 | 百度在线网络技术（北京）有限公司 | Information output method and device for automatic driving vehicle |
| CN108256577A (en)* | 2018-01-18 | 2018-07-06 | 东南大学 | A kind of barrier clustering method based on multi-line laser radar |
| CN108985171A (en)* | 2018-06-15 | 2018-12-11 | 上海仙途智能科技有限公司 | Estimation method of motion state and state estimation device |
| CN109360239A (en)* | 2018-10-24 | 2019-02-19 | 长沙智能驾驶研究院有限公司 | Obstacle detection method, device, computer equipment and storage medium |
| CN109344804A (en)* | 2018-10-30 | 2019-02-15 | 百度在线网络技术（北京）有限公司 | A kind of recognition methods of laser point cloud data, device, equipment and medium |
| CN110068814A (en)* | 2019-03-27 | 2019-07-30 | 东软睿驰汽车技术（沈阳）有限公司 | A kind of method and device measuring obstacle distance |
| Title |
|---|
| A combined clustering and image mapping based point cloud segmentation for 3D object detection;Fangchao Hu 等;2018 Chinese Control And Decision Conference (CCDC);全文* |
| 基于3D激光雷达的无人水面艇海上目标检测;李小毛;张鑫;王文涛;瞿栋;祝川;;上海大学学报(自然科学版)(第01期);全文* |
| 基于信息融合的智能车障碍物检测方法;陆峰;徐友春;李永乐;王德宇;谢德胜;;计算机应用(第S2期);全文* |
| 基于点云图的农业导航中障碍物检测方法;姬长英;沈子尧;顾宝兴;田光兆;张杰;;农业工程学报(第07期);全文* |
| Publication number | Publication date |
|---|---|
| CN112668371A (en) | 2021-04-16 |
| Publication | Publication Date | Title |
|---|---|---|
| CN115061386B (en) | Automated simulation test system and related equipment for intelligent driving | |
| CN114612616B (en) | Mapping method, device, electronic device and storage medium | |
| CN112622923B (en) | Method and apparatus for controlling a vehicle | |
| CN110390237A (en) | Processing Method of Point-clouds and system | |
| CN111353453B (en) | Obstacle detection method and device for vehicle | |
| CN109509236B (en) | Vehicle bounding box generation method and device in unmanned scene and storage medium | |
| CN109829447B (en) | Method and device for determining a three-dimensional frame of a vehicle | |
| CN110321854B (en) | Method and apparatus for detecting target object | |
| CN107515607A (en) | Control method and device for unmanned vehicle | |
| CN115240157A (en) | Road scene data persistence method, device, equipment and computer readable medium | |
| KR102463890B1 (en) | Method and apparatus for generating position information, device, media and program | |
| CN112558036B (en) | Method and device for outputting information | |
| CN112668371B (en) | Method and apparatus for outputting information | |
| CN110110696B (en) | Method and apparatus for processing information | |
| CN113361059A (en) | Vehicle simulation scene generation method and related equipment | |
| CN113126120B (en) | Data labeling method, device, equipment, storage medium and computer program product | |
| CN110120075B (en) | Method and apparatus for processing information | |
| CN112987707B (en) | A method and device for controlling automatic driving of a vehicle | |
| CN118585708A (en) | Intelligent driving function recommendation method, device, equipment and storage medium | |
| WO2024174723A1 (en) | Model training method and apparatus, and storage medium | |
| CN114724116B (en) | Vehicle traffic information generation method, device, equipment and computer readable medium | |
| CN113256715B (en) | Positioning method and device for robot | |
| CN113162809B (en) | Driving assistance processing method, driving assistance processing device, computer readable medium and electronic equipment | |
| CN116311155A (en) | Obstacle information generation method, obstacle information generation device, electronic device, and computer-readable medium | |
| CN111383337B (en) | Method and device for identifying objects |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |