技术领域Technical field
本申请涉及自动驾驶技术领域，特别是涉及一种自动驾驶车辆控制方法、装置、计算机设备和存储介质。The present application relates to the field of autonomous driving technology, and in particular, to an autonomous driving vehicle control method, device, computer equipment and storage medium.
背景技术Background technique
在自动驾驶领域，端到端自动驾驶是指使用深度神经网络技术对人的驾驶行为进行模仿和学习的一种自动驾驶技术，实现了传感信息到控制信息的直接映射，能形成完整的自动驾驶系统。In the field of autonomous driving, end-to-end autonomous driving refers to an autonomous driving technology that uses deep neural network technology to imitate and learn human driving behavior. It achieves direct mapping of sensing information to control information and can form a complete autonomous driving system. driving system.
目前，端到端自动驾驶系统的主体是一个神经网络，该神经网络在入口接收图像作为输入，在出口产生对车辆的控制指令作为输出，输入到输出直接映射，在此过程中，神经网络自适应地学习输入与输出之间的内在联系，形成一套封装严密的无人驾驶系统闭环，难以直接控制和干预系统内部的学习过程，且神经网络的学习能力有一定的限制，只能够处理简单问题，不能适应复杂的场景，例如在路口转向这一问题上，无人驾驶车辆面对复杂的场景，无法保证车辆在不同的路口以准确的角度转向，并顺利通过路口。At present, the main body of the end-to-end autonomous driving system is a neural network. The neural network receives images as input at the entrance and generates control instructions for the vehicle as output at the exit. The input is directly mapped to the output. In this process, the neural network automatically Adaptively learn the intrinsic relationship between input and output to form a tightly packaged closed loop of the unmanned driving system. It is difficult to directly control and intervene in the learning process within the system. Moreover, the learning ability of the neural network has certain limitations and can only handle simple tasks. The problem is that it cannot adapt to complex scenarios. For example, on the issue of turning at intersections, unmanned vehicles face complex scenarios and cannot guarantee that the vehicle will turn at accurate angles at different intersections and pass the intersection smoothly.
因此，现有的端到端自动驾驶系统存在无法适应复杂场景的技术问题。Therefore, existing end-to-end autonomous driving systems have technical problems that cannot adapt to complex scenarios.
发明内容Contents of the invention
基于此，有必要针对上述现有的端到端自动驾驶系统存在无法适应复杂场景的技术问题，提供一种自动驾驶车辆控制方法、装置、计算机设备和存储介质。Based on this, it is necessary to provide an automatic driving vehicle control method, device, computer equipment and storage medium in view of the above technical problem that the existing end-to-end automatic driving system cannot adapt to complex scenarios.
第一方面，本申请实施例提供一种自动驾驶车辆控制方法，所述方法包括：In a first aspect, embodiments of the present application provide an autonomous vehicle control method, which method includes:
获取车辆周围的当前环境信息；Obtain current environmental information around the vehicle;
将所述当前环境信息输入预设的导航分支神经网络，得到控制指令；所述导航分支神经网络为与预设的路口转向约束神经网络联合训练得到的网络模型；所述路口转向约束神经网络用于获取车辆与路肩之间的相对距离；The current environment information is input into the preset navigation branch neural network to obtain control instructions; the navigation branch neural network is a network model jointly trained with the preset intersection turning constraint neural network; the intersection turning constraint neural network is used To obtain the relative distance between the vehicle and the road shoulder;
根据所述控制指令控制所述车辆行驶。Control the driving of the vehicle according to the control instructions.
在其中一个实施例中，所述导航分支神经网络包括视觉特征提取神经网络和多个子分支神经网络；In one embodiment, the navigation branch neural network includes a visual feature extraction neural network and multiple sub-branch neural networks;
所述将所述当前环境信息输入预设的导航分支神经网络，得到控制指令，包括：The input of the current environment information into a preset navigation branch neural network to obtain control instructions includes:
采用所述视觉特征提取神经网络，从所述环境信息中提取视觉特征；Using the visual feature extraction neural network to extract visual features from the environmental information;
将所述视觉特征输入目标子分支神经网络，得到所述控制指令。The visual features are input into the target sub-branch neural network to obtain the control instructions.
在其中一个实施例中，所述子分支神经网络包括左转分支神经网络、右转分支神经网络和车道保持分支神经网络，则在所述将所述视觉特征输入目标子分支神经网络之前，所述方法包括：In one embodiment, the sub-branch neural network includes a left-turn branch neural network, a right-turn branch neural network and a lane keeping branch neural network, then before the visual features are input into the target sub-branch neural network, the The methods include:
获得车辆当前的导航信息，所述导航信息包括左转指令、右转指令和直行指令中任一个；Obtain the current navigation information of the vehicle, which includes any one of a left turn instruction, a right turn instruction, and a straight ahead instruction;
若所述导航信息为左转指令，则将所述左转分支神经网络确定为所述目标子分支神经网络；If the navigation information is a left turn instruction, determine the left turn branch neural network as the target sub-branch neural network;
若所述导航信息为右转指令，则将所述右转分支神经网络确定为所述目标子分支神经网络；If the navigation information is a right turn instruction, determine the right turn branch neural network as the target sub-branch neural network;
若所述导航信息为直行指令，则将所述车道保持分支神经网络确定为所述目标子分支神经网络。If the navigation information is a straight-going instruction, the lane keeping branch neural network is determined as the target sub-branch neural network.
在其中一个实施例中，所述方法还包括：In one embodiment, the method further includes:
将多组车辆周围的环境信息分别输入初始导航分支神经网络和初始路口转向约束神经网络中；Input the environmental information around multiple sets of vehicles into the initial navigation branch neural network and the initial intersection steering constraint neural network respectively;
将所述初始导航分支神经网络和所述初始路口转向约束神经网络的输出均输入预设的系统损失函数进行联合训练，得到所述系统损失函数的值；The outputs of the initial navigation branch neural network and the initial intersection turning constraint neural network are input into the preset system loss function for joint training to obtain the value of the system loss function;
根据所述系统损失函数的值，调整所述初始导航分支神经网络和所述初始路口转向约束神经网络的参数，直至所述系统损失函数的值达到预设阈值为止，得到所述导航分支神经网络和所述路口转向约束神经网络。According to the value of the system loss function, the parameters of the initial navigation branch neural network and the initial intersection turn constraint neural network are adjusted until the value of the system loss function reaches a preset threshold, and the navigation branch neural network is obtained and the intersection steering constraint neural network.
在其中一个实施例中，所述将所述初始导航分支神经网络和所述初始路口转向约束神经网络的输出均输入预设的系统损失函数进行联合训练，得到所述系统损失函数的值，包括：In one embodiment, the outputs of the initial navigation branch neural network and the initial intersection turning constraint neural network are input into a preset system loss function for joint training, and the value of the system loss function is obtained, including :
将所述初始导航分支神经网络的输出作为预设的第一损失函数的输入，将所述初始路口转向约束神经网络的输出作为预设的第二损失函数的输入，得到所述第一损失函数的值和所述第二损失函数的值；The output of the initial navigation branch neural network is used as the input of the preset first loss function, and the output of the initial intersection turn constraint neural network is used as the input of the preset second loss function to obtain the first loss function. The value of and the value of the second loss function;
根据所述第一损失函数的值、所述第一损失函数的权重、所述第二损失函数的值和所述第二损失函数的权重获取所述系统损失函数的值。The value of the system loss function is obtained according to the value of the first loss function, the weight of the first loss function, the value of the second loss function and the weight of the second loss function.
在其中一个实施例中，所述方法还包括：In one embodiment, the method further includes:
根据所述系统损失函数的值调整所述第一损失函数的权重和所述第二损失函数的权重。The weight of the first loss function and the weight of the second loss function are adjusted according to the value of the system loss function.
第二方面，本申请实施例提供一种自动驾驶车辆控制装置，所述装置包括：In a second aspect, embodiments of the present application provide an automatic driving vehicle control device, which includes:
环境信息获取模块，用于获取车辆周围的当前环境信息；Environmental information acquisition module, used to obtain the current environmental information around the vehicle;
控制指令输出模块，用于将所述当前环境信息输入预设的导航分支神经网络，得到控制指令；所述导航分支神经网络为与预设的路口转向约束神经网络联合训练得到的网络模型；所述路口转向约束神经网络用于获取车辆与路肩之间的相对距离；A control instruction output module is used to input the current environment information into a preset navigation branch neural network to obtain control instructions; the navigation branch neural network is a network model obtained by joint training with a preset intersection steering constraint neural network; so The intersection steering constraint neural network is used to obtain the relative distance between the vehicle and the road shoulder;
控制车辆行驶模块，用于根据所述控制指令控制所述车辆行驶。A vehicle travel control module is used to control the vehicle travel according to the control instructions.
第三方面，本申请实施例提供一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现上述第一方面任一实施例所述的步骤。In a third aspect, embodiments of the present application provide a computer device, including a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the method described in any embodiment of the first aspect. step.
第四方面，本申请实施例提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述第一方面任一实施例所述的步骤。In a fourth aspect, embodiments of the present application provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps described in any embodiment of the first aspect are implemented.
本申请实施例提供的一种自动驾驶车辆控制方法、装置、计算机设备和存储介质，计算机设备将获取的车辆周围当前的环境信息输入到预设的导航分支神经网络中，得到控制指令，并根据该控制指令控制车辆的行驶，由于该方法中，导航分支神经网络在训练时是联合路口约束神经网络一起进行训练的，在联合训练的过程中，路口约束神经网络可以辅助以及控制导航分支神经网络的学习，这样，即使遇到复杂的道路场景，导航分支神经网络也可以根据输出正确的控制指令，规范的控制自动驾驶车辆的行驶轨迹，保证车辆驾驶的安全性，大大提高了自动驾驶车辆对各种复杂场景的适应性。Embodiments of the present application provide an autonomous driving vehicle control method, device, computer equipment and storage medium. The computer equipment inputs the acquired current environmental information around the vehicle into a preset navigation branch neural network, obtains control instructions, and executes the control instructions according to the This control instruction controls the driving of the vehicle. In this method, the navigation branch neural network is trained together with the intersection constraint neural network during training. During the joint training process, the intersection constraint neural network can assist and control the navigation branch neural network. In this way, even when encountering complex road scenes, the navigation branch neural network can output correct control instructions and regulate the driving trajectory of the autonomous vehicle to ensure the safety of vehicle driving and greatly improve the accuracy of autonomous vehicles. Adaptability to various complex scenarios.
附图说明Description of the drawings
图1为一个实施例提供的一种自动驾驶车辆控制方法的应用环境图；Figure 1 is an application environment diagram of an autonomous vehicle control method provided in one embodiment;
图2为一个实施例提供的一种自动驾驶车辆控制方法的流程示意图；Figure 2 is a schematic flowchart of an autonomous vehicle control method provided in one embodiment;
图3为一个实施例提供的一种自动驾驶车辆控制方法的流程示意图；Figure 3 is a schematic flowchart of an autonomous vehicle control method provided in one embodiment;
图4为一个实施例提供的一种自动驾驶车辆控制方法的流程示意图；Figure 4 is a schematic flowchart of an autonomous vehicle control method provided in one embodiment;
图5为一个实施例提供的一种自动驾驶车辆控制方法的流程示意图；Figure 5 is a schematic flowchart of an autonomous vehicle control method provided in one embodiment;
图6为一个实施例提供的一种自动驾驶车辆控制方法的流程示意图；Figure 6 is a schematic flowchart of an autonomous vehicle control method provided in one embodiment;
图7为一个实施例提供的一种自动驾驶车辆控制系统的结构框图；Figure 7 is a structural block diagram of an autonomous vehicle control system provided in one embodiment;
图8为一个实施例提供的一种自动驾驶车辆控制装置的结构框图；Figure 8 is a structural block diagram of an autonomous vehicle control device provided in one embodiment;
图9为一个实施例提供的一种自动驾驶车辆控制装置的结构框图；Figure 9 is a structural block diagram of an autonomous vehicle control device provided in one embodiment;
图10为一个实施例提供的一种自动驾驶车辆控制装置的结构框图；Figure 10 is a structural block diagram of an autonomous vehicle control device provided in one embodiment;
图11为一个实施例提供的一种自动驾驶车辆控制装置的结构框图；Figure 11 is a structural block diagram of an autonomous vehicle control device provided in one embodiment;
图12为一个实施例提供的一种自动驾驶车辆控制装置的结构框图；Figure 12 is a structural block diagram of an autonomous vehicle control device provided in one embodiment;
图13为一个实施例提供的一种计算机设备内部结构图。Figure 13 is an internal structure diagram of a computer device according to an embodiment.
具体实施方式Detailed ways
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.
本申请提供的一种自动驾驶车辆控制方法，可以应用于如图1所示的应用环境中，该自动驾驶系统包括输入数据采集装置、计算机设备和车辆，其中，数据采集装置和计算机设备可以设置在车辆上，也可以设置在车辆以外，其中数据采集装置用于收集车辆周围当前环境信息，计算机设备用于根据数据采集装置收集的数据控制车辆驾驶，其中，该计算机设备包括通过系统总线连接的处理器、存储器、网络接口和数据库。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机程序和数据库。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的数据库用于存储自动驾驶车辆控制数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种自动驾驶车辆控制方法。An automatic driving vehicle control method provided by this application can be applied in the application environment as shown in Figure 1. The automatic driving system includes an input data collection device, a computer device and a vehicle, where the data collection device and the computer device can be configured On the vehicle or outside the vehicle, the data acquisition device is used to collect current environmental information around the vehicle, and the computer device is used to control vehicle driving based on the data collected by the data acquisition device. The computer device includes a computer connected through a system bus. Processor, memory, network interface and database. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile storage media and internal memory. The non-volatile storage medium stores operating systems, computer programs and databases. This internal memory provides an environment for the execution of operating systems and computer programs in non-volatile storage media. The computer device's database is used to store autonomous vehicle control data. The network interface of the computer device is used to communicate with external terminals through a network connection. The computer program, when executed by a processor, implements an autonomous vehicle control method.
本申请的实施例提供一种自动驾驶车辆控制方法、装置、计算机设备和存储介质，旨在解决现有的端到端自动驾驶系统存在无法适应复杂场景的技术问题。下面将通过实施例并结合附图具体地对本申请的技术方案以及本申请的技术方案如何解决上述技术问题进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。需要说明的是，本申请提供的一种自动驾驶车辆控制方法，图2-图6的执行主体为计算机设备，其执行主体还可以是自动驾驶车辆控制装置，其中该装置可以通过软件、硬件或者软硬件结合的方式实现成为自动驾驶车辆控制的部分或者全部。Embodiments of the present application provide an autonomous driving vehicle control method, device, computer equipment, and storage medium, aiming to solve the technical problem that existing end-to-end autonomous driving systems cannot adapt to complex scenarios. The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below through embodiments and in conjunction with the accompanying drawings. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. It should be noted that in the automatic driving vehicle control method provided by this application, the execution subject of Figures 2 to 6 is a computer device, and the execution subject can also be an automatic driving vehicle control device, wherein the device can be through software, hardware or The combination of software and hardware realizes part or all of the control of autonomous vehicles.
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments These are part of the embodiments of this application, but not all of them.
在一个实施例中，图2提供了一种自动驾驶车辆控制方法，本实施例涉及的是计算机设备根据车辆周围的当前环境信息，采用预设的导航分支神经网络，得到控制指令，并根据该控制指令控制车辆行驶的具体过程。如图2所示，该方法包括：In one embodiment, Figure 2 provides a self-driving vehicle control method. This embodiment involves the computer device using a preset navigation branch neural network to obtain control instructions based on the current environmental information around the vehicle, and based on the The control instructions control the specific process of vehicle driving. As shown in Figure 2, the method includes:
S101，获取车辆周围的当前环境信息。S101. Obtain current environment information around the vehicle.
本实施例中，车辆周围的环境信息表示的是该车辆周围的路况信息以及障碍物信息，其中该路况信息可以包括左右邻车道上的车流量、前方分叉路口等信息，其中障碍物信息可以包括路边障碍物的尺寸、位置等信息，本实施例对该环境信息中包括的内容不做具体限定。在实际应用中，计算机设备获取车辆周围的当前环境信息的方式可以是通过摄像头采集车辆周围的环境影像，再将采集的环境影像分为一帧一帧的图像信息，然后根据该图像信息确定具体的环境信息；还可以是通过雷达等其他环境传感器获取对周围环境信息，本实施例对此不做限定。In this embodiment, the environmental information around the vehicle represents the road condition information and obstacle information around the vehicle, where the road condition information may include information such as traffic volume on the left and right adjacent lanes, forking intersections ahead, etc., where the obstacle information may include It includes information such as the size and location of roadside obstacles. This embodiment does not specifically limit the content included in the environmental information. In practical applications, the way the computer device obtains the current environmental information around the vehicle may be to collect environmental images around the vehicle through a camera, and then divide the collected environmental images into frame-by-frame image information, and then determine the specific information based on the image information. Environmental information; the surrounding environment information can also be obtained through other environmental sensors such as radar, which is not limited in this embodiment.
S102，将所述当前环境信息输入预设的导航分支神经网络，得到控制指令；所述导航分支神经网络为与预设的路口转向约束神经网络联合训练得到的网络模型；所述路口转向约束神经网络用于获取车辆与路肩之间的相对距离。S102, input the current environment information into the preset navigation branch neural network to obtain control instructions; the navigation branch neural network is a network model jointly trained with the preset intersection turning constraint neural network; the intersection turning constraint neural network The network is used to obtain the relative distance between the vehicle and the road shoulder.
本步骤中，控制指令表示的控制车辆安全、规范地行驶的指令，例如该控制指令可以是方向盘转角、油门踏板位置以及刹车力度等，还可以是包括其他控制参数，本实施例对此不做限定。则基于上述S101步骤中计算机设备获取的车辆周围的当前环境信息，将该当前环境信息输入到预设的导航分支神经网络，就可以得到对应的控制指令，其中，该导航分支神经网络为与预设的路口转向约束神经网络联合训练得到的网络模型，该路口转向约束神经网络为获取车辆与路肩之间相对距离的网络模型，该路口转向约束神经网络训练时可以是将车辆周围的环境信息作为输入，进行训练学习，然后输出车辆与路肩的相对距离，其中，计算机设备对导航分支神经网络为与路口转向约束神经进行联合训练具体过程会在后续实施例中进行详细说明，本实施例在此不再赘述，本步骤中，将导航分支神经网络为与路口转向约束神经网络联合训练，可以使路口转向约束神经网络辅助导航分支神经网路的学习，对导航分支神经网络的学习进行控制，使导航分支神经网络输出的控制指令更符合车辆的行驶路径规范。In this step, the control instruction represents an instruction to control the vehicle to drive safely and standardly. For example, the control instruction can be the steering wheel angle, accelerator pedal position, braking force, etc., or it can also include other control parameters. This embodiment does not do this. limited. Then based on the current environment information around the vehicle obtained by the computer device in the above step S101, the current environment information is input into the preset navigation branch neural network, and the corresponding control instructions can be obtained, wherein the navigation branch neural network is the same as the preset navigation branch neural network. The intersection turning constraint neural network is a network model obtained by joint training with the intersection turning constraint neural network. The intersection turning constraint neural network is a network model that obtains the relative distance between the vehicle and the road shoulder. When training the intersection turning constraint neural network, the environment information around the vehicle can be used as Input, conduct training and learning, and then output the relative distance between the vehicle and the road shoulder. The computer device jointly trains the navigation branch neural network with the intersection steering constraint nerve. The specific process will be explained in detail in subsequent embodiments. This embodiment is here No need to go into details. In this step, the navigation branch neural network is jointly trained with the intersection turn constraint neural network, so that the intersection turn constraint neural network can assist the learning of the navigation branch neural network and control the learning of the navigation branch neural network, so that The control instructions output by the navigation branch neural network are more in line with the vehicle's driving path specifications.
S103，根据所述控制指令控制所述车辆行驶。S103. Control the driving of the vehicle according to the control instruction.
基于上述S102步骤得到的控制指令，计算机设备根据该控制指令控制车辆行驶，示例地，以控制指令是方向盘转角、油门踏板位置以及刹车力度为例，计算机设备根据该控制指令，控制车辆的方向盘转角的度数与控制指令中给出的度数相同、油门踏板位置与控制指令中给出的位置相同以及刹车力度与控制指令中给出的力度相同即可。需要说明的是，在本实施例中，虽然路口约束神经网络只在网络模型训练时辅助导航分支神经网络学习，在导航分支神经网络根据车辆周围的当前环境信息输出控制指令时不做干预，但是计算机设备在根据导航分支神经网络输出的控制指令控制车辆行驶时，会参考路口约束神经网络输出的车辆与路肩的相对距离，保证车辆的行驶安全性和规范性。Based on the control instruction obtained in the above step S102, the computer device controls the driving of the vehicle according to the control instruction. For example, taking the control instruction as steering wheel angle, accelerator pedal position and braking force as an example, the computer device controls the steering wheel angle of the vehicle according to the control instruction. The degree is the same as the degree given in the control instruction, the accelerator pedal position is the same as the position given in the control instruction, and the braking force is the same as the force given in the control instruction. It should be noted that in this embodiment, although the intersection constraint neural network only assists the navigation branch neural network learning during network model training, and does not intervene when the navigation branch neural network outputs control instructions based on the current environmental information around the vehicle, but When the computer device controls vehicle driving according to the control instructions output by the navigation branch neural network, it will refer to the relative distance between the vehicle and the road shoulder output by the intersection constraint neural network to ensure the safety and standardization of vehicle driving.
本实施例提供的自动驾驶车辆控制方法，计算机设备将获取的车辆周围当前的环境信息输入到预设的导航分支神经网络中，得到控制指令，并根据该控制指令控制车辆的行驶，由于该方法中，导航分支神经网络在训练时是联合路口约束神经网络一起进行训练的，在联合训练的过程中，路口约束神经网络可以辅助以及控制导航分支神经网络的学习，这样，即使遇到复杂的道路场景，导航分支神经网络也可以根据输出正确的控制指令，规范的控制自动驾驶车辆的行驶轨迹，保证车辆驾驶的安全性，大大提高了自动驾驶车辆对各种复杂场景的适应性。In the automatic driving vehicle control method provided by this embodiment, the computer device inputs the acquired current environmental information around the vehicle into the preset navigation branch neural network, obtains the control instruction, and controls the driving of the vehicle according to the control instruction. Due to this method In the training, the navigation branch neural network is trained together with the intersection constraint neural network. During the joint training process, the intersection constraint neural network can assist and control the learning of the navigation branch neural network. In this way, even if complex roads are encountered Scenarios, the navigation branch neural network can also standardize the driving trajectory of autonomous vehicles based on outputting correct control instructions to ensure the safety of vehicle driving and greatly improve the adaptability of autonomous vehicles to various complex scenarios.
其中，计算机设备在当前环境信息输入预设的导航分支神经网络以得到控制指令的具体过程，本申请通过以下几个实施例进行说明。其中上述导航分支神经网络包括视觉特征提取神经网络和多个子分支神经网络，则在上述实施例的基础上，如图3所示，上述S102步骤包括：Among them, the specific process of the computer device inputting the current environment information into the preset navigation branch neural network to obtain the control instructions is explained in this application through the following embodiments. The above-mentioned navigation branch neural network includes a visual feature extraction neural network and multiple sub-branch neural networks. Based on the above embodiment, as shown in Figure 3, the above-mentioned S102 step includes:
S201，采用所述视觉特征提取神经网络，从所述环境信息中提取视觉特征。S201: Use the visual feature extraction neural network to extract visual features from the environmental information.
本实施例中，基于上述S101步骤中获取的车辆周围的当前环境信息，计算机设备采用视觉特征提取神经网络从该环境信息中提取视觉特征，其中该视觉特征表示的是车辆周围环境信息中各物体以及空间的点、边缘线、直线、曲线等信息，由于视觉特征提取神经网络为预先根据多个环境信息训练好的模型，在实际应用中，假设计算机设备获取的环境信息的形式为图像形式，则计算机设备直接将这些图像形式的环境信息输入到该视觉特征提取网络中，得到的输出结果即为提取的视觉特征。In this embodiment, based on the current environmental information around the vehicle obtained in the above step S101, the computer device uses a visual feature extraction neural network to extract visual features from the environmental information, where the visual features represent each object in the environmental information surrounding the vehicle. As well as information such as points, edge lines, straight lines, and curves in space. Since the visual feature extraction neural network is a model trained in advance based on multiple environmental information, in practical applications, it is assumed that the environmental information obtained by the computer device is in the form of an image. The computer device directly inputs the environmental information in the form of images into the visual feature extraction network, and the output result is the extracted visual feature.
S202，将所述视觉特征输入目标子分支神经网络，得到所述控制指令。S202: Input the visual features into the target sub-branch neural network to obtain the control instructions.
其中，需要说明的是，由于导航分支神经网络在根据车辆周围的当前环境信息得到控制指令时，是分了多个子分支神经网络进行的，即不同路况采用不同的子分支神经网络进行控制指令输出，而不同的子分支神经网络则是根据不同路况预先训练好的网络模型。因此本步骤中，基于上述S201步骤提取的视觉特征，计算机设备会将该视觉特征输入到目标子分支神经网络中，以得到适合当前情况的控制指令。Among them, it should be noted that when the navigation branch neural network obtains control instructions based on the current environmental information around the vehicle, it is divided into multiple sub-branch neural networks, that is, different sub-branch neural networks are used for different road conditions to output control instructions. , and different sub-branch neural networks are pre-trained network models based on different road conditions. Therefore, in this step, based on the visual features extracted in the above S201 step, the computer device will input the visual features into the target sub-branch neural network to obtain control instructions suitable for the current situation.
本实施例提供的自动驾驶车辆控制方法，计算机设备先采用视觉特征提取神经网络从环境信息中提取视觉特征，然后将该视觉特征输入到目标子分支神经网络中，以得到控制指令，由于目标子分支神经网络表示的是适合当前路况的神经网络，这样得到的控制指令即为规范的控制指令，且不通过的路况，选择的目标子分支神经网络不同，大大提高了自动驾驶车辆对各种复杂场景的智能适应性。In the automatic driving vehicle control method provided by this embodiment, the computer device first uses a visual feature extraction neural network to extract visual features from environmental information, and then inputs the visual features into the target sub-branch neural network to obtain control instructions. Since the target sub-branch The branched neural network represents a neural network suitable for the current road conditions. The control instructions obtained in this way are standardized control instructions, and the target sub-branch neural network selected is different for unpassed road conditions, which greatly improves the ability of autonomous vehicles to deal with various complex Intelligent adaptability to scenarios.
上述实施例中提及子分支神经网络包括左转分支神经网络、右转分支神经网络和车道保持分支神经网络三种，即不同的路况对应不同的目标子分支神经网络不同，则表示计算机设备将视觉特征输入到目标子分支神经网络之前，计算机设备需先确定目标子分支神经网络，则如图4所示，所述方法包括：In the above embodiment, it is mentioned that the sub-branch neural network includes three types: left-turn branch neural network, right-turn branch neural network and lane keeping branch neural network. That is, different road conditions correspond to different target sub-branch neural networks, which means that the computer equipment will Before visual features are input to the target sub-branch neural network, the computer device needs to first determine the target sub-branch neural network, as shown in Figure 4. The method includes:
S301，获取车辆当前的导航信息；所述导航信息包括左转指令、右转指令和直行指令中任一个。S301. Obtain the current navigation information of the vehicle; the navigation information includes any one of a left turn instruction, a right turn instruction and a straight forward instruction.
本实施例中，子分支神经网络是根据不同的路况进行区分的，因此计算机设备需要确定当前的路况，以当前的路况为路口场景下行驶轨迹的选择为例，则计算机设备获取车辆当前的导航信息，其中该导航信息中包括左转指令、右转指令和直行指令中任一个，然后计算机设备根据该导航信息即可确定在该路口，车辆的行驶轨迹应该是左转，还是右转，或者是直行。其中计算机设备获取车辆当前的导航信息可以是从车辆的导航装置中获取，本实施例对此不做限定。In this embodiment, the sub-branch neural network is distinguished according to different road conditions. Therefore, the computer device needs to determine the current road conditions. Taking the current road conditions as the selection of the driving trajectory in the intersection scene as an example, the computer device obtains the current navigation of the vehicle. Information, wherein the navigation information includes any one of a left turn instruction, a right turn instruction, and a straight line instruction. Then the computer device can determine based on the navigation information whether the vehicle's driving trajectory should turn left or right at the intersection, or It's straight ahead. The computer device may obtain the current navigation information of the vehicle from the navigation device of the vehicle, which is not limited in this embodiment.
S302，若所述导航信息为左转指令，则将所述左转分支神经网络确定为所述目标子分支神经网络。S302: If the navigation information is a left turn instruction, determine the left turn branch neural network as the target sub-branch neural network.
仍以路口的场景为例，本步骤中，导航信息中指示当前路口需要左转，则计算机设备将左转分支神经网络确定为目标子分支神经网络，并将S201步骤中提取的视觉特征输入到左转分支神经网络中，这样输出的控制指令即为指示车辆在路口安全、规范的左转的方向盘转角、油门踏板位置以及刹车力度。Still taking the intersection scene as an example, in this step, if the navigation information indicates that a left turn is required at the current intersection, the computer device determines the left-turn branch neural network as the target sub-branch neural network, and inputs the visual features extracted in step S201 into In the left-turn branch neural network, the control instructions output in this way are the steering wheel angle, accelerator pedal position and braking force that instruct the vehicle to turn left safely and standardizedly at the intersection.
S303、若所述导航信息为右转指令，则将所述右转分支神经网络确定为所述目标子分支神经网络。S303. If the navigation information is a right turn instruction, determine the right turn branch neural network as the target sub-branch neural network.
本步骤中，导航信息中指示当前路口需要右转，则计算机设备将右转分支神经网络确定为目标子分支神经网络，并将S201步骤中提取的视觉特征输入到右转分支神经网络中，这样输出的控制指令即为指示车辆在路口安全、规范的右转的方向盘转角、油门踏板位置以及刹车力度。In this step, if the navigation information indicates that a right turn is required at the current intersection, the computer device determines the right turn branch neural network as the target sub-branch neural network, and inputs the visual features extracted in step S201 into the right turn branch neural network, so that The output control instructions are the steering wheel angle, accelerator pedal position and braking force that instruct the vehicle to turn safely and standardized right at the intersection.
S304、若所述导航信息为直行指令，则将所述车道保持分支神经网络确定为所述目标子分支神经网络。S304. If the navigation information is a straight-going instruction, determine the lane keeping branch neural network as the target sub-branch neural network.
本步骤中，导航信息中指示当前路口需要直行，则计算机设备将车道保持分支神经网络确定为目标子分支神经网络，并将S201步骤中提取的视觉特征输入到车道保持分支神经网络中，这样输出的控制指令即为指示车辆在路口安全、规范的直行的方向盘转角、油门踏板位置以及刹车力度。In this step, if the navigation information indicates that you need to go straight at the current intersection, the computer device determines the lane keeping branch neural network as the target sub-branch neural network, and inputs the visual features extracted in step S201 into the lane keeping branch neural network, thus outputting The control instructions are the steering wheel angle, accelerator pedal position and braking force that instruct the vehicle to drive safely and standardly at the intersection.
本实施例提供的自动驾驶车辆控制方法，计算机设备根据获取的导航信息确定目标子分支神经网络，并将从当前环境信息中提取的视觉特征输入到该目标子分支神经网络中，以得到对应的控制指令，由于目标子分支神经网络表示的是适合当前路况的神经网络，这样得到的控制指令即为规范的控制指令，且不通过的路况，选择的目标子分支神经网络不同，大大提高了自动驾驶车辆对各种复杂场景的智能适应性。In the automatic driving vehicle control method provided by this embodiment, the computer device determines the target sub-branch neural network based on the acquired navigation information, and inputs the visual features extracted from the current environmental information into the target sub-branch neural network to obtain the corresponding Control instructions, since the target sub-branch neural network represents a neural network suitable for the current road conditions, the control instructions obtained in this way are standardized control instructions, and for failed road conditions, the selected target sub-branch neural network is different, which greatly improves the automatic Intelligent adaptability of driving vehicles to various complex scenarios.
另外，本申请实施例还提供了一种自动驾驶车辆控制方法，其涉及的是计算机设备联合训练初始导航分支神经网络和初始路口转向约束神经网络，以得到上述导航分支神经网络和路口转向约束神经网络的具体过程，如图5所示，所述方法还包括：In addition, embodiments of the present application also provide an autonomous driving vehicle control method, which involves the joint training of an initial navigation branch neural network and an initial intersection steering constraint neural network by computer equipment to obtain the above-mentioned navigation branch neural network and intersection steering constraint neural network. The specific process of the network is shown in Figure 5. The method also includes:
S401，将多个车辆周围的环境信息分别输入初始导航分支神经网络和初始路口转向约束神经网络中。S401: Input the environmental information around multiple vehicles into the initial navigation branch neural network and the initial intersection steering constraint neural network respectively.
本实施例中，多个车辆周围的环境信息为预先采集的数据，则在实际应用中，计算机设备将该多个车辆周围的环境信息作为输入数据，分别输入到初始导航分支神经网络和初始路口约束神经网络中，以对初始导航分支神经网络和初始路口约束神经网络进行训练。In this embodiment, the environmental information around multiple vehicles is pre-collected data. In practical applications, the computer device uses the environmental information around multiple vehicles as input data and inputs it into the initial navigation branch neural network and the initial intersection respectively. In the constrained neural network, the initial navigation branch neural network and the initial intersection constrained neural network are trained.
S402，将所述初始导航分支神经网络和所述初始路口转向约束神经网络的输出均输入预设的系统损失函数进行联合训练，得到所述系统损失函数的值。S402: Input the outputs of the initial navigation branch neural network and the initial intersection turning constraint neural network into a preset system loss function for joint training to obtain the value of the system loss function.
基于上述S401步骤中，计算机设备可以得到初始导航分支神经网络和初始路口转向约束神经网络输出，然后将该输出均输入到预设系统损失函数中，得到系统损失函数的值，以达到对初始导航分支神经网络和初始路口约束神经网络进行联合训练，其中，系统损失函数的值用于评价初始导航分支神经网络和初始路口约束神经网络共同的训练结果。Based on the above S401 step, the computer device can obtain the output of the initial navigation branch neural network and the initial intersection steering constraint neural network, and then input the outputs into the preset system loss function to obtain the value of the system loss function to achieve the initial navigation The branch neural network and the initial intersection constraint neural network are jointly trained, in which the value of the system loss function is used to evaluate the joint training results of the initial navigation branch neural network and the initial intersection constraint neural network.
其中，如图6所示，本S402步骤的一种可实现方式包括：Among them, as shown in Figure 6, one possible implementation method of step S402 includes:
S501，将所述初始导航分支神经网络的输出作为预设的第一损失函数的输入，将所述初始路口转向约束神经网络的输出作为预设的第二损失函数的输入，得到所述第一损失函数的值和所述第二损失函数的值。S501, use the output of the initial navigation branch neural network as the input of the preset first loss function, use the output of the initial intersection turn constraint neural network as the input of the preset second loss function, and obtain the first The value of the loss function and the value of the second loss function.
本实施例中，计算机设备将初始导航分支神经网络的输出作为预设的第一损失函数的入，得到第一损失函数的值，将初始路口转向约束神经网络的输出作为预设的第二损失函数的输入，得到第二损失函数的值。In this embodiment, the computer device uses the output of the initial navigation branch neural network as the input of the preset first loss function to obtain the value of the first loss function, and uses the output of the initial intersection turn constraint neural network as the preset second loss. The input of the function gets the value of the second loss function.
S502，根据所述第一损失函数的值、所述第一损失函数的权重、所述第二损失函数的值和所述第二损失函数的权重获取所述系统损失函数的值。S502: Obtain the value of the system loss function according to the value of the first loss function, the weight of the first loss function, the value of the second loss function, and the weight of the second loss function.
基于上述S501步骤中得到的第一损失函数的值和第二损失函数的值，计算机设备根据该第一损失函数的值、第一损失函数的权重、第二损失函数的值和第二损失函数的权重获取系统损失函数的值，其中，该第一损失函数的权重和第二损失函数的权重初始可以是人为根据经验进行分配，例如：设定系统损失函数为L、第一损失函数为L1、第二损失函数为L2，则L＝αL1+βL2,其中α、β代表L1、L2在系统损失函数中的权重，α+β＝1，在训练过程中，该权重会根据训练结果进行调整，则可选地，计算机设备根据系统损失函数的值调整第一损失函数的权重和第二损失函数的权重，计算机设备对第一损失函数和第二损失函数的权重实时进行调整，以保证系统损失函数达到预设阈值。Based on the value of the first loss function and the value of the second loss function obtained in the above step S501, the computer device calculates the value of the first loss function according to the value of the first loss function, the weight of the first loss function, the value of the second loss function and the second loss function. The weight of the system loss function is obtained, wherein the weight of the first loss function and the weight of the second loss function can be initially allocated manually based on experience, for example: set the system loss function to be L and the first loss function to be L1 , the second loss function is L2, then L=αL1+βL2, where α and β represent the weights of L1 and L2 in the system loss function, α+β=1, during the training process, the weight will be adjusted according to the training results , then optionally, the computer device adjusts the weight of the first loss function and the weight of the second loss function according to the value of the system loss function, and the computer device adjusts the weight of the first loss function and the second loss function in real time to ensure that the system The loss function reaches a preset threshold.
S403，根据所述系统损失函数的值，调整所述初始导航分支神经网络和所述初始路口转向约束神经网络的参数，直至所述系统损失函数的值达到预设阈值为止，得到所述导航分支神经网络和所述路口转向约束神经网络。S403, adjust the parameters of the initial navigation branch neural network and the initial intersection turning constraint neural network according to the value of the system loss function until the value of the system loss function reaches a preset threshold, and obtain the navigation branch Neural network and the intersection steering constraint neural network.
本步骤中，基于上述S402步骤中得到的系统损失函数的值，计算机设备调整初始导航分支神经网络和初始路口约束神经网络的参数，以减少系统损失函数的值，直至该系统损失函数的值达到了预设阈值为止，表示初始导航分支神经网络和初始路口转向约束神经网络已训练好，即得到了上述导航分支神经网络和路口转向约束神经网络。In this step, based on the value of the system loss function obtained in the above step S402, the computer device adjusts the parameters of the initial navigation branch neural network and the initial intersection constraint neural network to reduce the value of the system loss function until the value of the system loss function reaches reaches the preset threshold, it means that the initial navigation branch neural network and the initial intersection turning constraint neural network have been trained, that is, the above navigation branch neural network and intersection turning constraint neural network are obtained.
本实施例提供的自动驾驶车辆控制方法，计算机设备将多个车辆周围的环境信息作为输入数据输入到初始导航分支神经网络和初始路口转向约束神经网络中，再将输出结果均输入到预设系统损失函数中，得到系统损失函数的值，然后根据该系统损失函数的值，调整该初始导航分支神经网络和初始路口转向约束神经网络的参数，直至系统损失函数的值达到预设阈值为止，即得到了训练好的导航分支神经网络和路口转向约束神经网络，这样，将初始导航分支神经网络和初始路口转向约束神经网络联合训练，采用系统损失函数统一对训练结果进行评价，实现路口转向约束神经网络辅助和控制导航分支神经网路的学习，使得导航分支神经网络即使遇到复杂的道路场景，也可以输出正确的控制指令，大大保证了自动驾驶车辆行驶的安全性和规范性。In the automatic driving vehicle control method provided by this embodiment, the computer device inputs environmental information around multiple vehicles as input data into the initial navigation branch neural network and the initial intersection steering constraint neural network, and then inputs the output results into the preset system In the loss function, the value of the system loss function is obtained, and then according to the value of the system loss function, the parameters of the initial navigation branch neural network and the initial intersection steering constraint neural network are adjusted until the value of the system loss function reaches the preset threshold, that is, The trained navigation branch neural network and intersection turn constraint neural network are obtained. In this way, the initial navigation branch neural network and the initial intersection turn constraint neural network are jointly trained, and the system loss function is used to uniformly evaluate the training results and realize the intersection turn constraint neural network. The learning of network-assisted and controlled navigation branch neural networks enables the navigation branch neural network to output correct control instructions even when encountering complex road scenes, greatly ensuring the safety and standardization of autonomous vehicle driving.
为进一步帮助理解，如图7所示，本申请实施例提供了一种中的自动驾驶车辆控制系统的结构框图，其中[1]摄像头作为自动驾驶车辆控制系统中的视觉输入端，获取行车过程中的前方图像。该图像作为输入，进入[2]视觉特征提取神经网络。[2]视觉特征提取神经网络使用卷积神经网络，对[1]中输入图像的特征进行提取，通过卷积神经网络对图像的学习能力，提取对驾驶过程中有用的视觉特征。[2]中提取的视觉特征为之后的各个子分支神经网络共同使用。[3]导航信息分支点提供导航指令，其中包括三种导航指令:左转、右转、直行。[2]中提取的视觉特征，在这三种导航指令后分别接入三条分支神经网络：[4]左转分支神经网络、[5]右转分支神经网络、[6]车道保持分支神经网络。每一条分支神经网络在启用时，输出一套控制量，其中[4]左转分支神经网络对应[7]控制，[5]右转分支神经网络对应[8]控制，[6]车道保持分支神经网络对应[9]控制。[7][8][9]控制中的控制量均包括方向盘转角、油门踏板位置以及刹车力度等，这样，通过执行该控制量，可以实现对自动驾驶车辆的控制。另外，路口转向约束神经网络使用[1]摄像头的图像作为输入，使用[11]神经网络进行学习，输出[12]车辆与路肩的相对距离。需要说明的是，导航分支网神经络和路口转向约束神经网络在训练过程中，会分别产生一套损失函数[10]损失函数1和[13]损失函数2，使用[10]损失函数1和[13]损失函数2，建立起[14]系统损失函数，这样，在本系统的训练过程中，以[14]系统损失函数为目标函数，通过网络的学习降低[14]系统损失函数的损失值，达到联合训练导航分支网神经络和路口转向约束神经网络的目的。To further assist understanding, as shown in Figure 7, an embodiment of the present application provides a structural block diagram of an autonomous vehicle control system, in which [1] the camera serves as the visual input terminal in the autonomous vehicle control system to obtain the driving process. front image in . This image is used as input into the visual feature extraction neural network [2]. [2] The visual feature extraction neural network uses a convolutional neural network to extract the features of the input image in [1]. Through the convolutional neural network's ability to learn the image, it can extract visual features that are useful in the driving process. The visual features extracted in [2] are commonly used by subsequent sub-branch neural networks. [3] The navigation information branch point provides navigation instructions, including three navigation instructions: turn left, turn right, and go straight. The visual features extracted in [2] are connected to three branch neural networks after these three navigation instructions: [4] Left turn branch neural network, [5] Right turn branch neural network, [6] Lane keeping branch neural network . When each branch neural network is enabled, it outputs a set of control quantities, among which [4] left-turn branch neural network corresponds to [7] control, [5] right-turn branch neural network corresponds to [8] control, [6] lane keeping branch Neural network corresponds to [9] control. [7][8][9] The control variables in the control include steering wheel angle, accelerator pedal position, braking force, etc. In this way, by executing the control variables, the control of the autonomous vehicle can be achieved. In addition, the intersection steering constraint neural network uses [1] camera images as input, uses [11] neural network to learn, and outputs [12] the relative distance between the vehicle and the road shoulder. It should be noted that during the training process, the navigation branch network neural network and the intersection turn constraint neural network will generate a set of loss functions [10] loss function 1 and [13] loss function 2 respectively. Use [10] loss function 1 and [13] Loss function 2 establishes the [14] system loss function. In this way, during the training process of this system, the [14] system loss function is used as the objective function, and the loss of the [14] system loss function is reduced through network learning. value, achieving the purpose of jointly training the navigation branch network neural network and the intersection steering constraint neural network.
应该理解的是，虽然图2-6的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图2-6中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。It should be understood that although the various steps in the flowcharts of Figures 2-6 are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in Figures 2-6 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. These sub-steps or stages The order of execution is not necessarily sequential, but may be performed in turn or alternately with other steps or sub-steps of other steps or at least part of the stages.
在一个实施例中，如图8所示，提供了一种自动驾驶车辆控制装置，所述装置包括：环境信息获取模块10、控制指令输出模块11和控制车辆行驶模块12，其中，In one embodiment, as shown in Figure 8, an automatic driving vehicle control device is provided. The device includes: an environment information acquisition module 10, a control instruction output module 11 and a vehicle driving control module 12, wherein,
环境信息获取模块10，用于获取车辆周围的当前环境信息；The environmental information acquisition module 10 is used to obtain the current environmental information around the vehicle;
控制指令输出模块11，用于将所述当前环境信息输入预设的导航分支神经网络，得到控制指令；所述导航分支神经网络为与预设的路口转向约束神经网络联合训练得到的网络模型；所述路口转向约束神经网络用于获取车辆与路肩之间的相对距离；The control instruction output module 11 is used to input the current environment information into a preset navigation branch neural network to obtain control instructions; the navigation branch neural network is a network model obtained by joint training with a preset intersection steering constraint neural network; The intersection steering constraint neural network is used to obtain the relative distance between the vehicle and the road shoulder;
控制车辆行驶模块12，用于根据所述控制指令控制所述车辆行驶。The vehicle travel control module 12 is used to control the vehicle travel according to the control instructions.
上述实施例提供的一种自动驾驶车辆控制装置，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。The implementation principles and technical effects of the automatic driving vehicle control device provided by the above embodiments are similar to those of the above method embodiments, and will not be described again here.
在一个实施例中，如图9所示，提供了一种自动驾驶车辆控制装置，上述控制指令输出模块11包括：视觉特征提取单元111和控制指令输出单元112，其中，In one embodiment, as shown in Figure 9, an automatic driving vehicle control device is provided. The above-mentioned control instruction output module 11 includes: a visual feature extraction unit 111 and a control instruction output unit 112, wherein,
视觉特征提取单元111，用于采用所述视觉特征提取神经网络，从所述环境信息中提取视觉特征；The visual feature extraction unit 111 is configured to use the visual feature extraction neural network to extract visual features from the environmental information;
控制指令输出单元112，用于将所述视觉特征输入目标子分支神经网络，得到所述控制指令。The control instruction output unit 112 is used to input the visual features into the target sub-branch neural network to obtain the control instruction.
上述实施例提供的一种自动驾驶车辆控制装置，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。The implementation principles and technical effects of the automatic driving vehicle control device provided by the above embodiments are similar to those of the above method embodiments, and will not be described again here.
在一个实施例中，如图10所示，提供了一种自动驾驶车辆控制装置，所述装置包括：导航信息获取单元113、第一子分支确定单元114、第二子分支确定单元115和第三子分支确定单元116，其中，In one embodiment, as shown in Figure 10, an automatic driving vehicle control device is provided. The device includes: a navigation information acquisition unit 113, a first sub-branch determination unit 114, a second sub-branch determination unit 115 and a third sub-branch determination unit 114. Three sub-branch determination unit 116, wherein,
导航信息获取单元113，用于获取车辆当前的导航信息；所述导航信息包括左转指令、右转指令和直行指令中任一个；The navigation information acquisition unit 113 is used to acquire the current navigation information of the vehicle; the navigation information includes any one of a left turn instruction, a right turn instruction, and a straight line instruction;
第一子分支确定单元114，用于若所述导航信息为左转指令，则将所述左转分支神经网络确定为所述目标子分支神经网络；The first sub-branch determination unit 114 is configured to determine the left-turn branch neural network as the target sub-branch neural network if the navigation information is a left-turn instruction;
第二子分支确定单元115，用于若所述导航信息为右转指令，则将所述右转分支神经网络确定为所述目标子分支神经网络；The second sub-branch determination unit 115 is configured to determine the right-turn branch neural network as the target sub-branch neural network if the navigation information is a right-turn instruction;
第三子分支确定单元116，用于若所述导航信息为直行指令，则将所述车道保持分支神经网络确定为所述目标子分支神经网络。The third sub-branch determination unit 116 is configured to determine the lane keeping branch neural network as the target sub-branch neural network if the navigation information is a straight-going instruction.
上述实施例提供的一种自动驾驶车辆控制装置，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。The implementation principles and technical effects of the automatic driving vehicle control device provided by the above embodiments are similar to those of the above method embodiments, and will not be described again here.
在一个实施例中，如图11所示，提供了一种自动驾驶车辆控制装置，所述装置还包括：输入模块13、训练模块14和调整模块15，其中，In one embodiment, as shown in Figure 11, an automatic driving vehicle control device is provided, and the device further includes: an input module 13, a training module 14 and an adjustment module 15, wherein,
输入模块13，用于将多个车辆周围的环境信息分别输入初始导航分支神经网络和初始路口转向约束神经网络中；The input module 13 is used to input environmental information around multiple vehicles into the initial navigation branch neural network and the initial intersection steering constraint neural network respectively;
训练模块14，用于将所述初始导航分支神经网络和所述初始路口转向约束神经网络的输出均输入预设的系统损失函数进行联合训练，得到所述系统损失函数的值；The training module 14 is used to input the outputs of the initial navigation branch neural network and the initial intersection turning constraint neural network into a preset system loss function for joint training to obtain the value of the system loss function;
调整模块15，用于根据所述系统损失函数的值，调整所述初始导航分支神经网络和所述初始路口转向约束神经网络的参数，直至所述系统损失函数的值达到预设阈值为止，得到所述导航分支神经网络和所述路口转向约束神经网络。The adjustment module 15 is used to adjust the parameters of the initial navigation branch neural network and the initial intersection turning constraint neural network according to the value of the system loss function until the value of the system loss function reaches a preset threshold, and we obtain The navigation branch neural network and the intersection turning constraint neural network.
上述实施例提供的一种自动驾驶车辆控制装置，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。The implementation principles and technical effects of the automatic driving vehicle control device provided by the above embodiments are similar to those of the above method embodiments, and will not be described again here.
在一个实施例中，如图12所示，提供了一种自动驾驶车辆控制装置，上述训练模块14包括：第一确定单元141和第二确定单元142，其中，In one embodiment, as shown in Figure 12, an automatic driving vehicle control device is provided. The above-mentioned training module 14 includes: a first determination unit 141 and a second determination unit 142, wherein,
第一确定单元141，用于将所述初始导航分支神经网络的输出作为预设的第一损失函数的输入，将所述初始路口转向约束神经网络的输出作为预设的第二损失函数的输入，得到所述第一损失函数的值和所述第二损失函数的值；The first determination unit 141 is configured to use the output of the initial navigation branch neural network as the input of the preset first loss function, and use the output of the initial intersection turning constraint neural network as the input of the preset second loss function. , obtain the value of the first loss function and the value of the second loss function;
第二确定单元142，用于根据所述第一损失函数的值、所述第一损失函数的权重、所述第二损失函数的值和所述第二损失函数的权重获取所述系统损失函数的值。The second determination unit 142 is configured to obtain the system loss function according to the value of the first loss function, the weight of the first loss function, the value of the second loss function, and the weight of the second loss function. value.
上述实施例提供的一种自动驾驶车辆控制装置，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。The implementation principles and technical effects of the automatic driving vehicle control device provided by the above embodiments are similar to those of the above method embodiments, and will not be described again here.
在一个实施例中，提供了一种自动驾驶车辆控制装置，所述装置还用于根据所述系统损失函数的值调整所述第一损失函数的权重和所述第二损失函数的权重。In one embodiment, an automatic driving vehicle control device is provided, the device is further configured to adjust the weight of the first loss function and the weight of the second loss function according to the value of the system loss function.
上述实施例提供的一种自动驾驶车辆控制装置，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。The implementation principles and technical effects of the automatic driving vehicle control device provided by the above embodiments are similar to those of the above method embodiments, and will not be described again here.
关于自动驾驶车辆控制装置的具体限定可以参见上文中对于自动驾驶车辆控制方法的限定，在此不再赘述。上述自动驾驶车辆控制装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。For specific limitations on the automatic driving vehicle control device, please refer to the above limitations on the automatic driving vehicle control method, which will not be described again here. Each module in the above-mentioned autonomous vehicle control device can be implemented in whole or in part by software, hardware, and combinations thereof. Each of the above modules may be embedded in or independent of the processor of the computer device in the form of hardware, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
在一个实施例中，提供了一种计算机设备，该计算机设备可以是终端，其内部结构图可以如图13所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种自动驾驶车辆控制方法。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，该计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in Figure 13. The computer equipment includes a processor, memory, network interface, display screen and input device connected by a system bus. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile storage media and internal memory. The non-volatile storage medium stores operating systems and computer programs. This internal memory provides an environment for the execution of operating systems and computer programs in non-volatile storage media. The network interface of the computer device is used to communicate with external terminals through a network connection. The computer program, when executed by a processor, implements an autonomous vehicle control method. The display screen of the computer device may be a liquid crystal display or an electronic ink display. The input device of the computer device may be a touch layer covered on the display screen, or may be a button, trackball or touch pad provided on the computer device shell. , it can also be an external keyboard, trackpad or mouse, etc.
本领域技术人员可以理解，上述图13中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。Those skilled in the art can understand that the structure shown in Figure 13 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Specific computer equipment More or fewer components may be included than shown in the figures, or certain components may be combined, or may have a different arrangement of components.
在一个实施例中，提供了一种计算机设备，包括存储器和处理器，存储器中存储有计算机程序，该处理器执行计算机程序时实现以下步骤：In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements the following steps:
获取车辆周围的当前环境信息；Obtain current environmental information around the vehicle;
将所述当前环境信息输入预设的导航分支神经网络，得到控制指令；所述导航分支神经网络为与预设的路口转向约束神经网络联合训练得到的网络模型；所述路口转向约束神经网络用于获取车辆与路肩之间的相对距离；The current environment information is input into the preset navigation branch neural network to obtain control instructions; the navigation branch neural network is a network model jointly trained with the preset intersection turning constraint neural network; the intersection turning constraint neural network is used To obtain the relative distance between the vehicle and the road shoulder;
根据所述控制指令控制所述车辆行驶。Control the driving of the vehicle according to the control instructions.
上述实施例提供的一种计算机设备，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。The implementation principles and technical effects of the computer device provided by the above embodiment are similar to those of the above method embodiment, and will not be described again here.
在一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现以下步骤：In one embodiment, a computer-readable storage medium is provided with a computer program stored thereon. When the computer program is executed by a processor, the following steps are implemented:
获取车辆周围的当前环境信息；Obtain current environmental information around the vehicle;
将所述当前环境信息输入预设的导航分支神经网络，得到控制指令；所述导航分支神经网络为与预设的路口转向约束神经网络联合训练得到的网络模型；所述路口转向约束神经网络用于获取车辆与路肩之间的相对距离；The current environment information is input into the preset navigation branch neural network to obtain control instructions; the navigation branch neural network is a network model jointly trained with the preset intersection turning constraint neural network; the intersection turning constraint neural network is used To obtain the relative distance between the vehicle and the road shoulder;
根据所述控制指令控制所述车辆行驶。Control the driving of the vehicle according to the control instructions.
上述实施例提供的一种计算机可读存储介质，其实现原理和技术效果与上述方法实施例类似，在此不再赘述。The implementation principles and technical effects of the computer-readable storage medium provided by the above embodiments are similar to those of the above method embodiments, and will not be described again here.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage. In the media, when executed, the computer program may include the processes of the above method embodiments. Any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Synchlink DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。The technical features of the above embodiments can be combined in any way. To simplify the description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, all possible combinations should be used. It is considered to be within the scope of this manual.
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。The above-described embodiments only express several implementation modes of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all fall within the protection scope of the present application. Therefore, the scope of protection of this patent application should be determined by the appended claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910237979.7ACN111830949B (en) | 2019-03-27 | 2019-03-27 | Automatic driving vehicle control method, device, computer equipment and storage medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910237979.7ACN111830949B (en) | 2019-03-27 | 2019-03-27 | Automatic driving vehicle control method, device, computer equipment and storage medium |
| Publication Number | Publication Date |
|---|---|
| CN111830949A CN111830949A (en) | 2020-10-27 |
| CN111830949Btrue CN111830949B (en) | 2024-01-16 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910237979.7AActiveCN111830949B (en) | 2019-03-27 | 2019-03-27 | Automatic driving vehicle control method, device, computer equipment and storage medium |
| Country | Link |
|---|---|
| CN (1) | CN111830949B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113276863B (en)* | 2021-07-01 | 2022-09-13 | 浙江吉利控股集团有限公司 | Vehicle control method, apparatus, device, medium, and program product |
| CN113515812B (en)* | 2021-07-09 | 2024-11-26 | 东软睿驰汽车技术（沈阳）有限公司 | Automatic driving method, device, processing equipment and storage medium |
| CN116125980A (en)* | 2022-12-30 | 2023-05-16 | 中国联合网络通信集团有限公司 | Unmanned truck driving method and device, electronic equipment and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2009295270A1 (en)* | 2008-09-19 | 2010-03-25 | The University Of Sydney | A method and system of data modelling |
| CN102529963A (en)* | 2010-12-14 | 2012-07-04 | 上海摩西海洋工程有限公司 | Computer-aided driving system |
| WO2016130719A2 (en)* | 2015-02-10 | 2016-08-18 | Amnon Shashua | Sparse map for autonomous vehicle navigation |
| CN106355948A (en)* | 2015-07-17 | 2017-01-25 | 本田技研工业株式会社 | Turn predictions |
| CN106918342A (en)* | 2017-03-10 | 2017-07-04 | 广州汽车集团股份有限公司 | Automatic driving vehicle driving path localization method and alignment system |
| CN107272687A (en)* | 2017-06-29 | 2017-10-20 | 深圳市海梁科技有限公司 | A kind of driving behavior decision system of automatic Pilot public transit vehicle |
| CN107545232A (en)* | 2016-06-24 | 2018-01-05 | 福特全球技术公司 | Track detection system and method |
| WO2018055378A1 (en)* | 2016-09-21 | 2018-03-29 | Oxford University Innovation Limited | Autonomous route determination |
| WO2018075325A1 (en)* | 2016-10-17 | 2018-04-26 | Uber Technologies, Inc. | Neural network system for autonomous vehicle control |
| CN108459588A (en)* | 2017-02-22 | 2018-08-28 | 腾讯科技（深圳）有限公司 | Automatic Pilot method and device, vehicle |
| CN108657189A (en)* | 2018-06-22 | 2018-10-16 | 南京航空航天大学 | Automatic driving steering system and its control method based on BP neural network and safe distance line shifting conditions |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9989964B2 (en)* | 2016-11-03 | 2018-06-05 | Mitsubishi Electric Research Laboratories, Inc. | System and method for controlling vehicle using neural network |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2009295270A1 (en)* | 2008-09-19 | 2010-03-25 | The University Of Sydney | A method and system of data modelling |
| CN102529963A (en)* | 2010-12-14 | 2012-07-04 | 上海摩西海洋工程有限公司 | Computer-aided driving system |
| WO2016130719A2 (en)* | 2015-02-10 | 2016-08-18 | Amnon Shashua | Sparse map for autonomous vehicle navigation |
| CN107438754A (en)* | 2015-02-10 | 2017-12-05 | 御眼视觉技术有限公司 | Sparse maps for autonomous vehicle navigation |
| CN106355948A (en)* | 2015-07-17 | 2017-01-25 | 本田技研工业株式会社 | Turn predictions |
| CN107545232A (en)* | 2016-06-24 | 2018-01-05 | 福特全球技术公司 | Track detection system and method |
| WO2018055378A1 (en)* | 2016-09-21 | 2018-03-29 | Oxford University Innovation Limited | Autonomous route determination |
| WO2018075325A1 (en)* | 2016-10-17 | 2018-04-26 | Uber Technologies, Inc. | Neural network system for autonomous vehicle control |
| CN108459588A (en)* | 2017-02-22 | 2018-08-28 | 腾讯科技（深圳）有限公司 | Automatic Pilot method and device, vehicle |
| CN106918342A (en)* | 2017-03-10 | 2017-07-04 | 广州汽车集团股份有限公司 | Automatic driving vehicle driving path localization method and alignment system |
| CN107272687A (en)* | 2017-06-29 | 2017-10-20 | 深圳市海梁科技有限公司 | A kind of driving behavior decision system of automatic Pilot public transit vehicle |
| CN108657189A (en)* | 2018-06-22 | 2018-10-16 | 南京航空航天大学 | Automatic driving steering system and its control method based on BP neural network and safe distance line shifting conditions |
| Title |
|---|
| Brain-Inspired Cognitive Model With Attention for Self-Driving Cars;Shitao Chen等;《 IEEE Transactions on Cognitive and Developmental Systems》;第11卷(第1期);全文* |
| 应用于智能车的车道信息感知及增量建模技术研究;于泳;《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》(第12期);全文* |
| Publication number | Publication date |
|---|---|
| CN111830949A (en) | 2020-10-27 |
| Publication | Publication Date | Title |
|---|---|---|
| CN109583151B (en) | Vehicle trajectory prediction method and device | |
| CN110406530B (en) | An automatic driving method, apparatus, apparatus and vehicle | |
| DE102018106713B4 (en) | METHOD FOR CONTROLLING A VEHICLE LANE CHANGE, DEVICE FOR CONTROLLING A VEHICLE LANE CHANGE AND ASSOCIATED DEVICES | |
| DE112017006567B4 (en) | SYSTEM AND METHOD FOR PROVIDING DRIVER TRAINING TO A HUMAN OCCUPANT OF AN AUTONOMOUS VEHICLE | |
| US12077176B2 (en) | Vehicle control method, apparatus, and device | |
| EP3588226B1 (en) | Method and arrangement for generating control commands for an autonomous road vehicle | |
| CN111830949B (en) | Automatic driving vehicle control method, device, computer equipment and storage medium | |
| DE102020106176A1 (en) | LOCALIZATION SYSTEMS AND PROCEDURES | |
| US12019414B2 (en) | Method for generating a training data set for training an artificial intelligence module for a control device of a vehicle | |
| EP3667556A1 (en) | Autonomous lane change | |
| CN109765902B (en) | Unmanned vehicle driving reference line processing method and device and vehicle | |
| WO2022052856A1 (en) | Vehicle-based data processing method and apparatus, computer, and storage medium | |
| CN109976334B (en) | Vehicle lane changing method, device, equipment and storage medium | |
| CN114644016B (en) | Vehicle automatic driving decision method and device, vehicle-mounted terminal and storage medium | |
| CN113383283A (en) | Perception information processing method and device, computer equipment and storage medium | |
| Gutiérrez et al. | Validation method of a self-driving architecture for unexpected pedestrian scenario in CARLA simulator | |
| CN114604268A (en) | Vehicle driving intention prediction method and device, electronic equipment and vehicle | |
| CN114019947B (en) | Method and system for controlling vehicle to travel at intersection and computer readable storage medium | |
| CN112507857B (en) | Lane line updating method, device, equipment and storage medium | |
| CN111923919A (en) | Vehicle control method, vehicle control device, computer equipment and storage medium | |
| US20240409122A1 (en) | Method for planning a trajectory of a driving maneuver of a motor vehicle, computer program product, computer-readable storage medium, and vehicle | |
| US12283088B2 (en) | Method and system for determining a driving corridor | |
| CN113954835B (en) | Method and system for controlling vehicle to travel at intersection and computer readable storage medium | |
| CN117128976B (en) | Method and device for acquiring road center line, vehicle and storage medium | |
| CN118182515B (en) | Vehicle lane-changing decision method, device and storage medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |