CN111433565A - Method and system for self-performance aware path planning in autonomous vehicles - Google Patents

Abstract

The present teachings relate to methods, systems, media, and embodiments for path planning for autonomous vehicles. First, an origin position and a destination position are obtained, where the destination position is a place to which the autonomous vehicle is going to drive. One or more available paths between the origin location and the destination location are identified. Based on the one or more available paths, a self-awareness performance model is instantiated that predicts an operating performance of the autonomous vehicle based on each of the one or more available paths. A planned path to the destination location is then automatically selected for the autonomous vehicle based on the self-awareness performance model.

Description

Translated fromChinese

用于自动驾驶车辆中的自身性能觉知路径规划的方法和系统Method and system for self-performance-aware path planning in autonomous vehicles

相关申请的交叉引用CROSS-REFERENCE TO RELATED APPLICATIONS

本申请与2017年12月18日提交的美国专利申请15/845,173相关，在此引入其整个内容作为参考。This application is related to US Patent Application 15/845,173, filed December 18, 2017, the entire contents of which are incorporated herein by reference.

技术领域technical field

本示教一般涉及自动驾驶。具体而言，本示教涉及自动驾驶中的规划和控制。This teach generally relates to autonomous driving. Specifically, this teaching relates to planning and control in autonomous driving.

背景技术Background technique

随着人工智能(AI)近来的技术进步，出现了在不同应用领域中应用AI的浪潮。这包括自动驾驶领域，在该领域中，规划和控制是必不可少的。如图1(现有技术)所示，自动驾驶模块110包括规划模块120和车辆控制模块130。如图2所示，规划可包括路径规划、运动规划或行为规划。路径规划指的是基于特定的考虑来规划从原点到目的地的路径的工作。With the recent technological advancements in artificial intelligence (AI), there has been a wave of applying AI in different application fields. This includes the field of autonomous driving, where planning and control are essential. As shown in FIG. 1 (prior art), theautonomous driving module 110 includes aplanning module 120 and avehicle control module 130 . As shown in Figure 2, planning may include path planning, motion planning, or behavior planning. Path planning refers to the work of planning a path from an origin to a destination based on specific considerations.

运动规划一般可指规划车辆的运动以实现特定效果的工作。例如，车辆的运动可以以符合交通规则或安全的方式来规划。于是，运动规划是确定车辆需要做出何种运动来实现该目的。行为规划一般指规划在不同情况下车辆应当如何表现的工作，例如，在经过十字路口时的车辆行为、待在车道内或沿车道行驶的车辆行为、或是在转弯时的车辆行为。例如，在超过前方缓慢移动的车辆时，可以规划特定的车辆行为。行为规划和运动规划可能是相关的，例如，规划的车辆行为可能需要翻译为运动，以便实现该行为。Motion planning can generally refer to the work of planning the motion of a vehicle to achieve a specific effect. For example, the movement of the vehicle can be planned in a way that is compliant with traffic regulations or safe. Motion planning, then, is to determine what motion the vehicle needs to make to achieve this. Behavior planning generally refers to the work of planning how a vehicle should behave in different situations, such as when passing an intersection, staying in or along a lane, or turning a corner. For example, specific vehicle behavior can be planned when overtaking a slow-moving vehicle ahead. Behavior planning and motion planning may be related, for example, a planned vehicle behavior may need to be translated into motion in order to implement that behavior.

图1所示的车辆控制130可涉及多种方面的控制。如图3所示，车辆控制可涉及：例如，针对道路特有的控制，针对运动特有的控制，针对质量(mass)特有的控制，针对几何结构特有的控制，针对空气动力特有的控制，以及针对轮胎特有的控制。Thevehicle control 130 shown in FIG. 1 may involve various aspects of control. As shown in FIG. 3 , vehicle control may involve, for example, road-specific control, motion-specific control, mass-specific control, geometry-specific control, aerodynamic-specific control, and Tyre-specific controls.

图1中的周围环境信息可用于车辆规划。传统而言，周围环境信息100包括例如车辆的当前位置、预定目的地、和/或交通信息。例如，使用这样的周围环境信息，传统的规划模块120可设计从当前位置到目的地的路径的规划。路径规划中使用的已知标准可包括例如最短距离、最短时间、使用高速公路(highways)、使用地方道路、交通量等等。这样的标准可以基于已知的信息来应用，这些已知信息例如为各个路段的距离、与道路相关联的已知的交通量模式等等。The surrounding environment information in Figure 1 can be used for vehicle planning. Traditionally,ambient information 100 includes, for example, the vehicle's current location, intended destination, and/or traffic information. For example, using such ambient information, theconventional planning module 120 can design a plan for a path from a current location to a destination. Known criteria used in route planning may include, for example, shortest distance, shortest time, use of highways, use of local roads, traffic volume, and the like. Such criteria may be applied based on known information such as distances of various road segments, known traffic volume patterns associated with the roads, and the like.

规划模块120也可执行运动规划，传统而言，运动规划基于例如用于状态空间的快速遍历随机树(RRT)或用于环境建模的马尔可夫决策过程(MDP)。规划模块120可基于所规划的路径/运动来生成将要馈送给车辆控制模块130的规划数据，使得车辆控制模块130可运行为以所规划的方式来控制车辆。为了使车辆运动以执行该规划，于是，车辆控制模块130可以生成控制信号140，该信号可以被发送到车辆的不同部分，以实现所规划的车辆运动。按照传统，车辆控制基于通用车辆运动学模型和/或不同类型的反馈控制器来执行。Theplanning module 120 may also perform motion planning, which is traditionally based on, for example, a fast traversal random tree (RRT) for state space or a Markov decision process (MDP) for environment modeling. Theplanning module 120 may generate planning data to be fed to thevehicle control module 130 based on the planned path/movement so that thevehicle control module 130 may operate to control the vehicle in the planned manner. To move the vehicle to perform the planning, thevehicle control module 130 may then generatecontrol signals 140 that may be sent to different parts of the vehicle to effect the planned vehicle movement. Traditionally, vehicle control is performed based on a generic vehicle kinematics model and/or different types of feedback controllers.

每个人类驾驶者一般以不同的偏好对车辆进行不同的操作或控制。人类驾驶者还基于实时情况对车辆进行适应性操作，这些实时情况可能由于车辆自身的当前状况、外在环境状况(其限制车辆运行的能力)和/或车内乘员对当前车辆运动的反应或响应而发生。例如，在车内有孩子的情况下，人类驾驶者可能出于安全原因在下雪天选择避开(路径规划)弯曲的路径。当不同的乘员乘坐车辆时，人类驾驶者可能以不同的方式驾驶，从而保证乘员的舒适。尽管人类驾驶者一般通过大致保持在车道中间位置来沿车道行驶的方式控制车辆，该行为可能在面临右转弯时改变。在这种情况下，同一人类驾驶者可在车辆接近右转点时弯向车道的右侧。另外，不同的人类驾驶者可能以不同的方式弯向右侧。另外，车道变换行为也可能在不同的周围环境情况下基于不同的车辆而不同。现有技术不涉及这些问题，更未提供解决方案。Each human driver generally operates or controls the vehicle differently with different preferences. Human drivers also adapt the vehicle based on real-time conditions, which may be due to the current conditions of the vehicle itself, external environmental conditions (which limit the ability of the vehicle to operate), and/or the reactions of occupants to current vehicle motion or occurs in response. For example, with children in the car, a human driver may choose to avoid (path planning) a curved path on a snowy day for safety reasons. When different occupants ride in the vehicle, the human driver may drive in different ways to ensure the comfort of the occupants. Although a human driver typically controls the vehicle by staying roughly in the middle of the lane to drive along the lane, this behavior may change when faced with a right turn. In this case, the same human driver can swerve to the right of the lane as the vehicle approaches a right-turn point. Additionally, different human drivers may turn to the right in different ways. In addition, lane change behavior may also be different based on different vehicles under different ambient conditions. The prior art does not address these problems, nor does it provide solutions.

因此，存在提供在自动驾驶中用于规划和控制的改进方案的需求。Therefore, there is a need to provide improved solutions for planning and control in autonomous driving.

发明内容SUMMARY OF THE INVENTION

这里公开的示教涉及用于在线服务的方法、系统和程序设计。具体而言，本示教涉及用于开发能与用户对话的虚拟代理的方法、系统和程序设计。The teachings disclosed herein relate to methods, systems, and programming for online services. In particular, the present teachings relate to methods, systems, and programming for developing virtual agents that can converse with users.

在一实例中，公开了用于自动驾驶车辆的路径规划的方法。首先获得原点位置和目的地位置，其中，目的地位置是自动驾驶车辆将要去的地方。识别原点位置和目的地位置之间的一个或多于一个可用路径。自身性能觉知模型基于所述一个或多于一个可用路径得到实例化(instantiate)，且其预测自动驾驶车辆基于所述一个或多于一个可用路径中的各个的运行性能。于是，基于自身性能觉知模型，到目的地位置的规划路径被自动选择，用于自动驾驶车辆。In one example, a method for path planning of an autonomous vehicle is disclosed. First, the origin location and destination location are obtained, where the destination location is where the autonomous vehicle will go. Identify one or more available paths between the origin location and the destination location. The self-performance awareness model is instantiated based on the one or more available paths, and it predicts the operational performance of the autonomous vehicle based on each of the one or more available paths. Thus, based on the self-performance awareness model, the planned path to the destination location is automatically selected for the autonomous vehicle.

在另一实例中，公开了用于自动驾驶车辆的路径规划的系统。该系统包括接口单元、全局路径规划器和路径选择引擎。接口单元被配置为获得与原点位置以及目的地位置有关的信息，其中，目的地位置是自动驾驶车辆将要去的地方。全局路径规划器被配置为识别原点位置和目的地位置之间的一个或多于一个的可用路径。路径选择引擎被配置为获得基于所述一个或多于一个可用路径实例化的自身性能觉知模型，其中，自身性能觉知模型预测自动驾驶车辆基于所述一个或多于一个可用路径的运行性能，并从所述一个或多于一个可用路径中为自动驾驶车辆选择在原点位置和目的地位置之间的规划路径。In another example, a system for path planning of an autonomous vehicle is disclosed. The system includes an interface unit, a global path planner and a path selection engine. The interface unit is configured to obtain information about an origin location and a destination location, where the destination location is where the autonomous vehicle is going to go. The global path planner is configured to identify one or more available paths between the origin location and the destination location. The path selection engine is configured to obtain an ego performance awareness model instantiated based on the one or more available paths, wherein the ego performance awareness model predicts the operational performance of the autonomous vehicle based on the one or more available paths , and selects a planned path for the autonomous vehicle between the origin location and the destination location from the one or more available paths.

其他概念涉及用于实现关于开发虚拟代理的当前示教的软件。根据此概念的软件产品包括至少一个机器可读的非暂时性介质以及由该介质承载的信息。该介质承载的信息可以是可执行程序代码数据、与可执行程序代码相关联的参数和/或与用户、请求、内容相关的信息或与社会群体有关的信息，等等。Other concepts relate to software used to implement current teachings about developing virtual agents. A software product according to this concept includes at least one machine-readable non-transitory medium and information carried by the medium. The information carried by the medium may be executable program code data, parameters associated with the executable program code, and/or information related to users, requests, content, or social groups, and the like.

在一实例中，公开了机器可读的非暂时性介质，其中，该介质上存有与用于自动驾驶车辆的路径规划有关的信息，使得该信息在被机器读取时使该机器执行下面的操作步骤。首先，获得原点位置和目的地位置，其中，目的地位置是自动驾驶车辆将要去的地方。识别原点位置和目的地位置之间的一个或多于一个的可用路径。自身性能觉知模型基于所述一个或多一个的可用路径得到实例化，且其预测自动驾驶车辆基于所述一个或多于一个可用路径中的各个的运行性能。于是，基于自身性能觉知模型，为自动驾驶车辆自动选择到目的地位置的规划路径。In one example, a machine-readable non-transitory medium is disclosed, wherein the medium has information related to path planning for an autonomous vehicle stored thereon, such that the information, when read by a machine, causes the machine to perform the following: operation steps. First, an origin location and a destination location are obtained, where the destination location is where the autonomous vehicle will go. Identify one or more available paths between the origin location and the destination location. The self-performance awareness model is instantiated based on the one or more available paths, and it predicts the operational performance of the autonomous vehicle based on each of the one or more available paths. Therefore, based on the self-performance awareness model, the planned path to the destination location is automatically selected for the autonomous vehicle.

另外的新特征有部分在下面的说明书中给出，有部分将由本领域技术人员在检视下面的说明书和附图后明了或通过制造或运行实例来习得。本示教的新特征可以通过使用或实践下面讨论的具体实例中给出的方法、设备和组合的多种方面来获得。Additional novel features are set forth in part in the following description, and in part will be apparent to those skilled in the art upon review of the following description and drawings, or learned by manufacturing or operating examples. The novel features of the present teachings can be obtained by using or practicing various aspects of the methods, apparatus, and combinations presented in the specific examples discussed below.

附图说明Description of drawings

这里描述的方法、系统和/或程序设计进一步在示例性实施例中介绍。这些示例性实施例参照附图详细介绍。这些实施例是非限制性的示例性实施例，其中，贯穿附图中的几幅附图，类似的参考标号代表类似的结构，且其中：The methods, systems and/or programming described herein are further described in the exemplary embodiments. These exemplary embodiments are described in detail with reference to the accompanying drawings. These embodiments are non-limiting exemplary embodiments, wherein like reference numerals represent similar structures throughout the several figures of the drawing, and wherein:

图1(现有技术)示出了自动驾驶的某些重要模块；Figure 1 (prior art) shows some important modules of autonomous driving;

图2示出了自动驾驶中的规划的示例性类型；Figure 2 illustrates an exemplary type of planning in autonomous driving;

图3示出了公知类型的车辆控制；Figure 3 shows a vehicle control of a known type;

图4A示出了根据本示教一实施例具有规划模块和车辆控制模块的自动驾驶车辆；4A illustrates an autonomous vehicle having a planning module and a vehicle control module according to an embodiment of the present teachings;

图4B示出了根据本示教一实施例的实时数据的示例性类型；Figure 4B illustrates an exemplary type of real-time data according to an embodiment of the present teachings;

图5示出了根据本示教一实施例的规划模块的示例性高层次系统图；5 illustrates an exemplary high-level system diagram of a planning module according to an embodiment of the present teachings;

图6A示出了根据本示教一实施例用于实现自身觉知性能模型的示例性方法；6A illustrates an exemplary method for implementing a self-awareness performance model according to an embodiment of the present teachings;

图6B示出了根据本示教一实施例，具有参数的自身性能觉知模型的示例性构造；FIG. 6B illustrates an exemplary construction of a self-performance awareness model with parameters in accordance with an embodiment of the present teachings;

图6C示出了根据本示教一实施例的内在车辆性能参数的示例性类型；FIG. 6C illustrates exemplary types of intrinsic vehicle performance parameters in accordance with an embodiment of the present teachings;

图6D示出了根据本示教一实施例的外在性能参数的示例性类型；FIG. 6D illustrates exemplary types of extrinsic performance parameters according to an embodiment of the present teachings;

图7示出了根据本示教一实施例，用于生成将在规划时考虑的自身性能觉知参数的机制的示例性高层次系统图；7 illustrates an exemplary high-level system diagram of a mechanism for generating self-performance awareness parameters to be considered in planning, according to an embodiment of the present teachings;

图8示出了根据本示教一实施例的自身性能觉知参数生成器的示例性高层次系统图；8 illustrates an exemplary high-level system diagram of a self-performance awareness parameter generator according to an embodiment of the present teachings;

图9示出了根据本示教一实施例，用于生成自身性能觉知参数的示例性过程的流程图；9 shows a flowchart of an exemplary process for generating self-performance awareness parameters according to an embodiment of the present teachings;

图10示出了根据本示教一实施例的路径规划模块的示例性高层次系统图；10 illustrates an exemplary high-level system diagram of a path planning module according to an embodiment of the present teachings;

图11为根据本示教一实施例的用于路径规划的示例性过程图的流程图；11 is a flowchart of an exemplary process diagram for path planning according to an embodiment of the present teachings;

图12示出了根据本示教一实施例的全局路径规划器的示例性高层次系统图；12 illustrates an exemplary high-level system diagram of a global path planner in accordance with an embodiment of the present teachings;

图13为根据本示教一实施例的全局路径规划器的示例性过程的流程图；13 is a flowchart of an exemplary process of a global path planner according to an embodiment of the present teachings;

图14A示出了根据本示教一实施例的运动规划模块的示例性高层次系统图；14A illustrates an exemplary high-level system diagram of a motion planning module in accordance with an embodiment of the present teachings;

图14B示出了根据本示教一实施例的乘员模块的示例性类型；Figure 14B illustrates an exemplary type of passenger module in accordance with an embodiment of the present teachings;

图14C示出了根据本示教一实施例的示例性类型的将要为运动规划观察的用户反应；14C illustrates an exemplary type of user response to be observed for motion planning in accordance with an embodiment of the present teachings;

图15示出了根据本示教一实施例的乘员观察分析器的示例性高层次系统图；15 illustrates an exemplary high-level system diagram of an occupant observation analyzer according to an embodiment of the present teachings;

图16为根据本示教一实施例的乘员观察分析器的示例性过程的流程图；16 is a flow diagram of an exemplary process for an occupant observation analyzer in accordance with an embodiment of the present teachings;

图17为根据本示教一实施例的运动规划模块的示例性过程的流程图；17 is a flowchart of an exemplary process of a motion planning module according to an embodiment of the present teachings;

图18示出了根据本示教一实施例，用于生成运动规划的不同模型的模型训练机制的示例性高层次系统图；18 shows an exemplary high-level system diagram of a model training mechanism for generating different models of motion planning according to an embodiment of the present teachings;

图19示出了根据本示教一实施例的不同类型的将被观察的反应及其在模型训练中的作用；Figure 19 illustrates different types of responses to be observed and their role in model training according to an embodiment of the present teachings;

图20A示出了根据本示教一实施例的车道相关规划的示例性类型；FIG. 20A illustrates an exemplary type of lane-dependent planning in accordance with an embodiment of the present teachings;

图20B示出了根据本示教一实施例的与车道沿行(lane following)有关的行为的示例性类型；20B illustrates an exemplary type of behavior related to lane following according to an embodiment of the present teachings;

图20C示出了根据本示教一实施例的与车道变换有关的行为的示例性类型；FIG. 20C illustrates an exemplary type of behavior related to a lane change in accordance with an embodiment of the present teachings;

图21示出了根据本示教一实施例的车道规划模块的示例性高层次系统图；21 illustrates an exemplary high-level system diagram of a lane planning module in accordance with an embodiment of the present teachings;

图22为根据本示教一实施例的车道规划模块的示例性过程的流程图；22 is a flowchart of an exemplary process of a lane planning module in accordance with an embodiment of the present teachings;

图23A示出了根据车辆运动学模型生成车辆控制信号的传统方法；Figure 23A shows a conventional method of generating vehicle control signals from a vehicle kinematics model;

图23B示出了根据本示教一实施例，使得似人型车辆控制成为可能的车辆控制模块的高层次系统图；23B shows a high-level system diagram of a vehicle control module that enables humanoid vehicle control according to an embodiment of the present teachings;

图23C示出了根据本示教一实施例，使得个性化似人型车辆控制成为可能的车辆控制模块的高层次系统图；23C shows a high-level system diagram of a vehicle control module that enables personalized humanoid vehicle control according to an embodiment of the present teachings;

图24示出了根据本示教一实施例的似人型车辆控制单元的示例性高层次系统图；24 illustrates an exemplary high-level system diagram of a humanoid vehicle control unit in accordance with an embodiment of the present teachings;

图25为根据本示教一实施例的似人型车辆控制单元的示例性过程的流程图；25 is a flowchart of an exemplary process of a humanoid vehicle control unit in accordance with an embodiment of the present teachings;

图26为根据本示教一实施例的似人型车辆控制模型生成器的示例性高层次系统图；26 is an exemplary high-level system diagram of a humanoid vehicle control model generator in accordance with an embodiment of the present teachings;

图27为根据本示教一实施例的似人型车辆控制模型生成器的示例性过程的流程图；27 is a flowchart of an exemplary process of a humanoid vehicle control model generator in accordance with an embodiment of the present teachings;

图28为根据本示教一实施例的似人型车辆控制信号生成器的示例性高层次系统图；28 is an exemplary high-level system diagram of a humanoid vehicle control signal generator in accordance with an embodiment of the present teachings;

图29为根据本示教一实施例的似人型车辆控制信号生成器的示例性过程的流程图；29 is a flowchart of an exemplary process for a humanoid vehicle control signal generator in accordance with an embodiment of the present teachings;

图30示出了一移动装置的架构，该装置可用于实现包括本示教的专用系统；以及Figure 30 illustrates the architecture of a mobile device that can be used to implement a dedicated system including the present teachings; and

图31示出了一计算机的架构，该计算机可用于实现包括本示教的专用系统。Figure 31 shows the architecture of a computer that can be used to implement a special purpose system that includes the present teachings.

具体实施方式Detailed ways

在下面的具体介绍中，通过举例的方式给出了多种具体细节，以便提供对相关示教的全面理解。然而，本领域技术人员应当明了，本示教可以在没有这些细节的情况下实现。在其他的实例中，公知的方法、过程、部件和/或电路以相对较高的层次介绍而没有细节，以便避免不必要地模糊本示教的实施形态。In the following detailed description, various specific details are given by way of example in order to provide a thorough understanding of the relevant teachings. However, it will be apparent to those skilled in the art that the present teachings may be practiced without these details. In other instances, well-known methods, procedures, components and/or circuits have been described at a relatively high level without detail in order to avoid unnecessarily obscuring the embodiments of the present teachings.

本公开一般涉及系统、方法、介质和其他的实施形态，其用于以适应实时情况的自身性能觉知式的、似人型、个性化的方式来规划和控制路径/车辆行为。图4A示出了根据本示教一实施例，具有车辆规划/控制机制410的自动车辆。自动车辆规划/控制机制410包括规划模块440和车辆控制模块450。两个模块均将多种类型的信息用作输入，以便实现自身性能觉知、似人型、个性化且适应于实时情况的运行。例如，如所示出的，规划模块440和车辆控制模块450均接收历史人工(manual)驾驶数据430，以便习得似人型的在不同情况下操纵车辆的方法。这些模块还接收实时数据480，从而觉知车辆周围的动态情况，以便相应地对运行进行适应。另外，规划模块440访问自身性能觉知模块490，该模块描绘在车辆当前所处的情况下是什么在限制车辆的运行性能。The present disclosure generally relates to systems, methods, media, and other implementations for planning and controlling path/vehicle behavior in an own-performance-aware, human-like, personalized manner that adapts to real-time situations. FIG. 4A shows an autonomous vehicle with a vehicle planning/control mechanism 410 in accordance with an embodiment of the present teachings. The automatic vehicle planning/control mechanism 410 includes aplanning module 440 and avehicle control module 450 . Both modules use multiple types of information as input in order to perform self-aware, human-like, personalized, and real-time situations. For example, as shown, both theplanning module 440 and thevehicle control module 450 receive historicalmanual driving data 430 in order to learn a human-like approach to maneuvering the vehicle in different situations. These modules also receive real-time data 480 to be aware of the dynamics around the vehicle to adapt operation accordingly. Additionally, theplanning module 440 accesses the self-performance awareness module 490, which depicts what is limiting the vehicle's operational performance under the conditions the vehicle is currently in.

实时数据480可包括对于车辆规划和控制有用或与之相关的不同类型的信息。图4B示出了根据本发明一实施例的实时数据的示例性类型。例如，示例性实时数据可包括车辆相关数据、时间相关数据、乘员相关数据、天气相关数据、……、以及与附近的道路有关的数据。车辆相关数据可包括，例如，车辆在该时刻的运动状态、位置或状况。车辆的运动状态可涉及，例如，其当前速度和行驶方向。实时位置信息可包括，例如，车辆的当前纬度、经度和高度。车辆的实时状况可包括：车辆的功能状态，例如，车辆当前是处于完全还是部分功能状态；或者，特定参数，车辆的不同部件正在该参数下运行；等等。Real-time data 480 may include different types of information useful or relevant to vehicle planning and control. Figure 4B illustrates an exemplary type of real-time data according to an embodiment of the present invention. For example, exemplary real-time data may include vehicle-related data, time-related data, occupant-related data, weather-related data, . . . , and data related to nearby roads. Vehicle-related data may include, for example, the state of motion, location, or condition of the vehicle at that moment. The vehicle's motion state may relate, for example, to its current speed and direction of travel. Real-time location information may include, for example, the vehicle's current latitude, longitude, and altitude. The real-time conditions of the vehicle may include: the functional state of the vehicle, eg, whether the vehicle is currently in a fully or partially functional state; or, specific parameters under which different components of the vehicle are operating; and the like.

时间相关的实时数据一般可包括当前日期、时间或月份。乘员相关数据可包括与车辆的乘员有关的多种特性，例如：乘员反应线索，其可包括从乘员观察到的视觉、听觉或行为线索；或者，乘员的状况，例如乘员的精神状态、身体状态或功能状态。乘员的状况可以基于从乘员反应线索观察到的线索来推断。天气相关数据可包括车辆当前位于的当地的天气。道路相关数据可包括：与附近的道路的物理状况有关的信息，例如，道路的弯曲度、陡度或湿度；或者，本地交通状况，例如沿该道路的拥堵情况。Time-related real-time data may generally include the current date, time, or month. Occupant-related data may include various characteristics related to the occupant of the vehicle, such as: occupant response cues, which may include visual, auditory, or behavioral cues observed from the occupant; or, the condition of the occupant, such as the occupant's mental state, physical state or functional status. The condition of the occupant can be inferred based on the clues observed from the occupant response clues. The weather-related data may include the local weather in which the vehicle is currently located. Road-related data may include information about the physical conditions of nearby roads, such as the curvature, steepness, or humidity of the road; or, local traffic conditions, such as congestion along the road.

图5示出了根据本示教一实施例的规划模块440的示例性高层次系统图。在此示例性实施例中，规划包括但不限于道路规划、运动规划以及车道相关行为的规划，其包括车道沿行、车道变换等等。相应地，在此所示实施例中，规划模块440包括路径规划模块550、运动规划模块560以及车道规划模块570。各个模块的目标在于以自身性能觉知式的、似人型的且个性化的方式运行。除了周围环境信息420外，模块550、560和570中的每一个将记录的人类驾驶数据430、实时数据480和自身性能觉知模型490用作输入，并生成将由车辆控制模块450使用的其相应的输出，以便变换为控制车辆的车辆控制信号470。例如，路径规划模块550生成作为其输出的规划路径信息520，运动规划模块560生成作为其输出的规划运动530，车道规划模块570生成作为其输出的规划车道控制信息540。FIG. 5 illustrates an exemplary high-level system diagram ofplanning module 440 in accordance with an embodiment of the present teachings. In this exemplary embodiment, planning includes, but is not limited to, road planning, motion planning, and planning of lane-related behavior, including lane following, lane changes, and the like. Accordingly, in the embodiment shown here, theplanning module 440 includes apath planning module 550 , amotion planning module 560 , and alane planning module 570 . The goal of each module is to operate in a performance-aware, human-like, and individualized manner. In addition toambient information 420 , each ofmodules 550 , 560 , and 570 uses recordedhuman driving data 430 , real-time data 480 , and self-performance awareness model 490 as input, and generates its corresponding output for transformation into avehicle control signal 470 that controls the vehicle. For example,path planning module 550 generates as its output plannedpath information 520,motion planning module 560 generates as its output plannedmotion 530, andlane planning module 570 generates as its output plannedlane control information 540.

规划模块中的每一个可以经由某触发信号来触发。例如，路径规划模块550可经由路径规划触发信号来致动；运动规划模块560可在接收到运动规划触发信号时致动；而车道规划模块570可在车道规划触发信号被接收到时开始运行。这样的触发信号可以手动提供(由例如驾驶者或乘员)或基于例如特定配置或特定事件自动生成。驾驶者可手动致动路径规划模块550或任何其他的用于路径/运动/车道规划的规划模块，很像是人们手动启动例如车辆巡航控制时所做的那样。Each of the planning modules can be triggered via some trigger signal. For example,path planning module 550 may be activated via a path planning trigger signal;motion planning module 560 may be activated upon receipt of a motion planning trigger signal; andlane planning module 570 may begin operation upon receipt of a lane planning trigger signal. Such trigger signals may be provided manually (eg, by the driver or occupant) or automatically generated based on, eg, specific configurations or specific events. The driver may manually actuate thepath planning module 550 or any other planning module for path/motion/lane planning, much as one would manually activate eg vehicle cruise control.

规划活动也可由特定的配置或事件来致动。例如，车辆可以被配置为，每当车辆接收到指示下一个目的地的输入时，致动路径规划。这可以与车辆当前位置处于哪里无关。在某些实施例中，规划模块可以在每当车辆开启时总是触发，并且，依赖于情况，它们可以按照需要从事不同的规划活动。在不同的情况下，它们还可以以该情况需要的方式彼此交互。例如，车道规划模块570可在特定情况下判断为变换车道。这样的规划车道控制由车道规划模块570输出，并可馈送到运动规划模块560，使得适合用于实现规划车道变换的特定路径轨迹(规划运动)可以进一步由运动规划模块560进行规划。Planning activities can also be activated by specific configurations or events. For example, the vehicle may be configured to activate path planning each time the vehicle receives an input indicating a next destination. This may be independent of where the vehicle's current location is. In some embodiments, the planning modules may always be triggered whenever the vehicle is turned on, and, depending on the situation, they may engage in different planning activities as needed. In different situations, they can also interact with each other in the way that the situation requires. For example, thelane planning module 570 may determine to change lanes under certain circumstances. Such planned lane control is output bylane planning module 570 and can be fed tomotion planning module 560 so that specific path trajectories (planned motions) suitable for implementing planned lane changes can be further planned bymotion planning module 560 .

规划模块的输出可被馈送到规划模块440中的另一个，用于进一步规划或是用于提供用于另一模块的未来规划的输入。例如，路径规划模块550的输出(规划路径520)可被馈送到运动规划模块560，使得路径信息可影响车辆运动如何被规划。如上面所讨论的，车道规划模块570的输出(规划车道控制540)可被馈送到运动规划模块560，使得规划的车道控制行为可经由规划的运动控制来实现。相反，运动规划模块560的输出(规划运动530)也可被馈送到车道规划模块570，从而影响车道控制行为的规划。例如，在个性化运动规划中，运动规划模块560可判断为，由于观察到车辆乘员偏好平稳运动，车辆的运动需要轻柔。这样的判断是运动规划的一部分，并可被发送到车道规划模块570，使得车辆的车道控制行为可以以确保平稳运动的方式(例如，尽可能少地变换车道)进行。The output of the planning modules may be fed to another of theplanning modules 440 for further planning or to provide input for future planning of the other module. For example, the output of path planning module 550 (planned path 520) may be fed tomotion planning module 560 so that the path information may affect how vehicle motion is planned. As discussed above, the output of lane planning module 570 (planned lane control 540 ) can be fed tomotion planning module 560 so that planned lane control behavior can be implemented via planned motion control. Conversely, the output of the motion planning module 560 (planned motion 530) may also be fed to thelane planning module 570, thereby affecting the planning of lane control actions. For example, in personalized motion planning, themotion planning module 560 may determine that the motion of the vehicle needs to be gentle due to the observation that vehicle occupants prefer smooth motion. Such determinations are part of motion planning and can be sent to thelane planning module 570 so that the vehicle's lane control behavior can be made in a manner that ensures smooth motion (eg, changing lanes as little as possible).

为了确保车辆行为以自身性能觉知的方式受到规划和控制，路径规划模块550、运动规划模块560和车道规划模块570也访问自身性能觉知模块490，并将之用于以这样的方式确定规划策略：将当前场景内，车辆实际能够做什么考虑在内。图6A示出了根据本示教一实施例实现自身性能觉知模型490的示例性方式。如所示的，自身性能觉知模型490可被构建为概率模型、参数模型或描述性模型。这样的模型可基于例如学习来训练。模型可包括多种参数，这些用于描绘可影响或对车辆的实际能力有影响的因素。模型可以实现为概率模型，其具有概率性推定的参数。模型还可实现为参数模型，其具有适用于不同的真实世界状况的显式模型属性。模型490也可被提供为具有枚举状况的描述性模型，这些状况具有基于实时场景实例化的值。To ensure that vehicle behavior is planned and controlled in an ego-aware manner,path planning module 550,motion planning module 560, andlane planning module 570 also access ego-awareness module 490 and use it to determine the plan in this manner Strategy: Take into account what the vehicle can actually do in the current scene. FIG. 6A illustrates an exemplary manner of implementing a self-performance awareness model 490 in accordance with an embodiment of the present teachings. As shown, self-performance awareness model 490 may be constructed as a probabilistic model, a parametric model, or a descriptive model. Such models can be trained based on, for example, learning. The model may include a variety of parameters that characterize factors that may affect or have an effect on the actual capabilities of the vehicle. The model may be implemented as a probabilistic model with probabilistically inferred parameters. Models can also be implemented as parametric models, which have explicit model properties suitable for different real-world situations.Model 490 may also be provided as a descriptive model with enumerated conditions having values instantiated based on real-time scenarios.

任何情况下的自身性能觉知模型490可包括多种参数，其中的每一个与可能影响车辆实际性能的某些参数相关联，使得车辆规划(路径、运动或车辆)必须考虑。在下面的公开中，自身性能觉知模型和自身性能觉知参数可互换地使用。图6B示出了根据本示教一实施例的自身性能觉知模型或参数510的示例性构造。如所示的，自身性能觉知参数510可包括内在性能参数和外在性能参数。内在车辆性能参数可以指与车辆自身相关联的参数，其可影响车辆在运行中能够做什么，且这样的参数可以基于车辆如何制造或车辆在该时刻是什么样来确定。外在性能参数可以指这样的周围环境参数或特性：其处于车辆的外部，但可能影响车辆能被运行的方式。The self-performance awareness model 490 in any case may include a variety of parameters, each of which is associated with certain parameters that may affect the actual performance of the vehicle, such that vehicle planning (path, motion, or vehicle) must be considered. In the following disclosure, self-performance awareness models and self-performance awareness parameters are used interchangeably. FIG. 6B illustrates an exemplary construction of a self-performance awareness model orparameter 510 in accordance with an embodiment of the present teachings. As shown, self-performance awareness parameters 510 may include intrinsic performance parameters and extrinsic performance parameters. Intrinsic vehicle performance parameters may refer to parameters associated with the vehicle itself that may affect what the vehicle is capable of doing in operation, and such parameters may be determined based on how the vehicle is built or what the vehicle is like at that moment in time. Extrinsic performance parameters may refer to ambient environment parameters or characteristics that are external to the vehicle but may affect the way the vehicle can be operated.

图6C示出了根据本示教一实施例的内在车辆性能参数的示例性类型。如图所示，内在车辆性能参数可包括、但不限于车辆在例如其发动机、其安全措施及其轮胎等方面的特性。例如，在其发动机方面，内在性能参数可具体指定车辆能达到的最大速度、能在发动机上进行的控制，包括巡航控制或任何对发动机手动控制的限制。在安全措施方面，内在性能参数可包括关于车辆配有何种传感器的信息、与制动器(breaks)有关的具体参数或与车辆座位有关的信息。例如，某些车辆可具有用金属支撑物(更强)作为后背的座位，某些座位仅仅具有塑料支撑物。某些座位可具有允许自动控制以便进行振动的机制，某些没有。在车辆其他部件方面，内在性能参数也可具体指定车辆轮胎的类型(其可具有可进行操作的轴承)以及车辆当前是否装有雪胎或配有防滑措施。这些内在车辆性能参数可用于评估何种类型的路径和运动可能是可行的以及哪些类型的车辆行为可以实现。因此，使这样的内在性能参数可用于规划模块允许规划模块适当地进行规划，而不超出车辆实际能够做到的范围。FIG. 6C illustrates exemplary types of intrinsic vehicle performance parameters in accordance with an embodiment of the present teachings. As shown, intrinsic vehicle performance parameters may include, but are not limited to, characteristics of the vehicle such as its engine, its safety features, and its tires. For example, in terms of its engine, intrinsic performance parameters may specify the maximum speed the vehicle can achieve, the controls that can be performed on the engine, including cruise control or any limitations on manual engine control. In terms of safety measures, intrinsic performance parameters may include information about what sensors the vehicle is equipped with, specific parameters related to brakes, or information related to vehicle seating. For example, some vehicles may have seats with metal supports (stronger) as backs, and some seats only have plastic supports. Some seats may have mechanisms that allow automatic control to vibrate, some do not. Among other components of the vehicle, intrinsic performance parameters may also specify the type of vehicle tires (which may have operable bearings) and whether the vehicle is currently fitted with snow tires or with anti-skid measures. These intrinsic vehicle performance parameters can be used to assess what types of paths and motions are likely to be feasible and what types of vehicle behaviors can be achieved. Thus, making such intrinsic performance parameters available to the planning module allows the planning module to plan appropriately without going beyond what the vehicle is actually capable of.

图6D示出了根据本示教一实施例的外在性能参数的示例性类型。如上面所讨论的，外在性能参数具体指定处于车辆外部但可能影响规划能力的信息，且这样的外在性能参数用于在给定车辆外部状况的情况下确定合适的规划。来自规划模块的最终输出可以在内在和外在性能参数的双重限制下确定。由于外在性能参数可包括描述车辆处于或可能面对的状况或情况的参数，它们可能将会影响应当规划什么。例如，与道路(接近车辆或甚至是距离车辆相对较远)有关的周围环境情况。道路状况相关参数可指示道路有多拥挤(故行驶速度不能规划得太快)、道路是否有速度限制(规定的最小和最大速度，以及由于交通量导致的实际可允许速度)、沿着道路是否存在任何事故、或道路的表面状况(故运动不能太快)、或道路表面是否存在将会在车辆规划中妨碍车辆性能的特定状况。FIG. 6D illustrates exemplary types of extrinsic performance parameters in accordance with an embodiment of the present teachings. As discussed above, extrinsic performance parameters specify information that is external to the vehicle but that may affect planning capabilities, and such extrinsic performance parameters are used to determine an appropriate plan given vehicle external conditions. The final output from the planning module can be determined under the dual constraints of intrinsic and extrinsic performance parameters. Since extrinsic performance parameters may include parameters that describe the conditions or situations that the vehicle is in or may face, they may affect what should be planned. For example, ambient conditions related to the road (approaching the vehicle or even relatively far from the vehicle). Parameters related to road conditions can indicate how congested the road is (so the driving speed cannot be planned too fast), whether the road has speed limits (specified minimum and maximum speeds, and the actual allowable speed due to traffic volume), whether along the road There are any accidents, or the surface condition of the road (so the movement cannot be too fast), or whether there are certain conditions on the road surface that will hinder the vehicle's performance in the vehicle planning.

还存在可能影响多种规划活动的其他车辆外部状况。这包括光照或大气相关状况，以及车辆的周围环境。例如，如果车辆的位置处于存在太阳眩光、故传感器不能良好运行，这将影响规划决策。如果车辆位于有者浓雾状况的区域之中，这样的信息对规划模块也是重要的。如果存在大量降水，这样的信息也可被规划模块考虑在内。周围的交通对于规划也可能很重要。例如，外在参数可提供与附近车辆或物体有关的信息，使得规划模块可在其相应的规划中考虑这些信息。外在参数可包括关于这些附近的车辆/物体的信息，例如，附近的车辆是大卡车还是自行车，其也可影响规划决策如何做出。另外，沿着车辆所处道路发生的事件也可影响规划。例如，出于明显的原因，对于规划模块来说，车辆当前是否处于学区中的道路上或沿着车辆当前所处道路是否有正在进行的施工也可能属于重要信息。There are also other vehicle external conditions that may affect a variety of planning activities. This includes lighting or atmospheric related conditions, as well as the vehicle's surroundings. For example, if the vehicle is in a position where there is sun glare so the sensors do not perform well, this will affect planning decisions. Such information is also important to the planning module if the vehicle is located in an area with dense fog conditions. If there is a lot of precipitation, such information can also be taken into account by the planning module. Surrounding traffic can also be important for planning. For example, extrinsic parameters can provide information about nearby vehicles or objects so that the planning module can take this information into account in its corresponding planning. Extrinsic parameters may include information about these nearby vehicles/objects, eg whether the nearby vehicle is a large truck or a bicycle, which may also influence how planning decisions are made. In addition, events along the road the vehicle is on can also affect planning. For example, whether the vehicle is currently on a road in a school zone or whether there is ongoing construction along the road the vehicle is currently on may also be important information to the planning module for obvious reasons.

外在性能参数可以在时间上连续获取和更新，以支持规划模块实时地基于外部情况对其决策进行适应。在某些情况下，也可预测外在性能参数。例如，如果车辆在下午在道路上向西行驶，可以预测将会有太阳眩光。尽管这种预测的外在性能参数可能不是实时信息，但其将帮助规划模块(例如路径规划模块)做出适当的决策。例如，如果车辆的预定目的地在西北方向，且该时刻向西和向北都有可用的道路，知道如果在傍晚向西走会有太阳眩光的情况下，路径规划模块550可相应地决定当时首先选择向北的道路，稍后在太阳下山后选择向西的道路，以避免太阳眩光(更为安全)。这种预测的外在性能参数可以基于其他的信息(例如车辆的当前位置和车辆的预定目的地)来确定。Extrinsic performance parameters can be continuously acquired and updated in time to support the planning module to adapt its decisions based on extrinsic conditions in real time. In some cases, extrinsic performance parameters can also be predicted. For example, if a vehicle is driving west on the road in the afternoon, it can be predicted that there will be sun glare. Although such predicted extrinsic performance parameters may not be real-time information, it will help planning modules (eg, path planning modules) to make appropriate decisions. For example, if the intended destination of the vehicle is in the northwest direction, and there are roads available to the west and north at that time, knowing that there will be sun glare if going westward in the evening, thepath planning module 550 can accordingly determine the first priority at that time. Choose the road to the north, and the road to the west later after the sun goes down to avoid sun glare (safer). Such predicted extrinsic performance parameters may be determined based on other information such as the vehicle's current location and the vehicle's intended destination.

采用性能参数(包括内在和外在二者)，车辆变得能够自身觉知内在和外在性能相关限制，这在规划方面可能非常重要。图7示出了根据本示教一实施例，用于生成自身觉知性能参数的机制700的示例性高层次系统图。在此所示实施例中，机制700包含当地背景确定单元730和自身觉知性能参数生成器740。基于例如关于车辆当前位置和/或车辆所前往目的地的信息，当地背景确定单元730将收集车辆所位于和/或将要位于的地方(即，车辆当前所在的地方和车辆在前往目的地的路上将要位于的地方)的局部信息。例如，基于与车辆有关的信息以及根据例如车辆当前和未来位置所确定的当地背景信息，自身觉知性能参数生成器740将连续地生成内在和外在性能参数二者。Using performance parameters (both intrinsic and extrinsic), the vehicle becomes self-aware of intrinsic and extrinsic performance-related constraints, which can be very important in terms of planning. FIG. 7 shows an exemplary high-level system diagram of amechanism 700 for generating self-awareness performance parameters in accordance with an embodiment of the present teachings. In the embodiment shown here, themechanism 700 includes a localcontext determination unit 730 and a self-awarenessperformance parameter generator 740 . Based on, for example, information about the vehicle's current location and/or the destination the vehicle is traveling to, the localcontext determination unit 730 will collect where the vehicle is and/or will be located (ie, where the vehicle is currently and the vehicle is on the way to the destination). where it will be located) local information. For example, the self-awareperformance parameter generator 740 will continuously generate both intrinsic and extrinsic performance parameters based on vehicle-related information and local contextual information determined from, for example, the vehicle's current and future location.

为了促进自身性能觉知参数生成器740生成外在性能参数，当地背景确定单元730可基于当前位置720和目的地信息710来取得存储在地图配置750和道路背景配置760中的信息。与道路有关的当地背景信息可包括车辆当前位于的道路和/或车辆后来将要位于的道路的周围环境或背景信息。例如，地图配置750可提供关于从当前位置到预定目的地的道路的信息，而道路背景配置760可提供关于与道路相关联的特性(例如各条道路的高度、陡度、弯曲度等等)的某些已知或静态信息。这些收集的关于道路的静态信息于是可由自身性能觉知参数生成器740使用。To facilitate self-performanceawareness parameter generator 740 to generate extrinsic performance parameters, localcontext determination unit 730 may retrieve information stored inmap configuration 750 androad context configuration 760 based oncurrent location 720 anddestination information 710 . The local contextual information about the road may include ambient or contextual information about the road on which the vehicle is currently located and/or the road on which the vehicle will be located later. For example, themap configuration 750 may provide information about the road from the current location to the intended destination, while theroad background configuration 760 may provide information about the characteristics associated with the road (eg, the height, steepness, curvature, etc. of each road) certain known or static information. These collected static information about the road can then be used by the self-performanceawareness parameter generator 740 .

道路状况可随时间变化。例如，道路可能由于天气状况的变化而变得多冰或变滑。关于道路的这样的动态变化的背景信息可由例如自身性能觉知参数生成器740连续地分别获得，并用于生成反映实时情况的外在性能参数。如将要在下面参照关于自身性能觉知参数生成器740的图8讨论的，当前位置和原点-目的地信息均可被发送到自身性能觉知参数生成器740，以便由其收集关于道路状况的实时信息，从而确定外在性能参数。Road conditions can change over time. For example, roads may become icy or slippery due to changing weather conditions. Background information about such dynamic changes of the road may be continuously separately obtained, eg, by the self-performance-awareness parameter generator 740, and used to generate extrinsic performance parameters that reflect real-time conditions. As will be discussed below with reference to FIG. 8 with respect to the self-performanceawareness parameter generator 740, both the current position and origin-destination information can be sent to the self-performanceawareness parameter generator 740 for the purpose of collecting information about road conditions therefrom. real-time information to determine extrinsic performance parameters.

为了生成内在车辆性能信息，可以从车辆信息存储器750访问与车辆有关的信息。车辆信息存储器750可存储在车辆制造时配置的车辆参数，例如车辆是否具有巡航控制或特定类型的传感器。存储器750也可在随后更新与车辆内在参数有关的信息。这样的随后更新可以由于例如车辆维护或修理或甚至是实时观察到的更新而生成。在下面参照图8的讨论中，自身性能觉知参数生成器740还包括这样的机制：其连续收集与车辆的实际内在性能一致的车辆相关参数的任何动态更新。To generate intrinsic vehicle performance information, vehicle-related information may be accessed from thevehicle information store 750 . Thevehicle information memory 750 may store vehicle parameters configured when the vehicle is manufactured, such as whether the vehicle has cruise control or certain types of sensors. Thememory 750 may also subsequently update information related to vehicle intrinsic parameters. Such subsequent updates may be generated due to, for example, vehicle maintenance or repairs or even updates observed in real time. In the discussion below with reference to FIG. 8, the self-performanceawareness parameter generator 740 also includes a mechanism that continuously collects any dynamic updates of vehicle-related parameters that are consistent with the vehicle's actual intrinsic performance.

图8示出了根据本示教一实施例的自身性能觉知参数生成器740的示例性高层次系统图。在此所示实施例中，自身性能觉知参数生成器740包括当地背景信息处理器810、情景参数确定器820、自身性能觉知参数更新器830以及连续且动态收集与做出车辆决策有关的不同方面的信息的多种更新器。这些动态信息更新器包括例如车辆性能参数更新器860-a、天气敏感参数更新器860-b、交通敏感参数更新器860-c、方位敏感参数更新器860-d、道路敏感参数更新器860-e、……、以及时间敏感参数更新器860-f。FIG. 8 shows an exemplary high-level system diagram of self-performanceawareness parameter generator 740 in accordance with an embodiment of the present teachings. In the embodiment shown here, the self-performanceawareness parameter generator 740 includes a local context information processor 810, acontextual parameter determiner 820, a self-performance awareness parameter updater 830, and a continuous and dynamic collection of data relevant to making vehicle decisions. Multiple updaters for different aspects of information. These dynamic information updaters include, for example, Vehicle Performance Parameter Updater 860-a, Weather Sensitive Parameter Updater 860-b, Traffic Sensitive Parameter Updater 860-c, Orientation Sensitive Parameter Updater 860-d, Road Sensitive Parameter Updater 860- e, ..., and time sensitive parameter updater 860-f.

在某些运行实施例中，在接收到来自当地背景确定单元730的当地背景信息时，当地背景信息处理器810处理所接收的信息，并且，例如，提取关于车辆位于的当前路径的信息，并将该信息发送到自身觉知性能参数更新器830。与当前路径有关的这样的信息可包括路径的陡度或弯曲度，或其他类型的静态信息，例如，路径的高度和方位。情景参数确定器820接收当前位置720，并且，例如，分离出位置和时间信息，并将信息发送到自身觉知性能参数识别器830，使得其可使用该信息来识别精确的时间以及位置特有的性能参数。In some operating embodiments, upon receiving local context information from the localcontext determination unit 730, the local context information processor 810 processes the received information and, for example, extracts information about the current path the vehicle is located on, and This information is sent to self-awareness performance parameter updater 830. Such information related to the current path may include the steepness or curvature of the path, or other types of static information, such as the height and bearing of the path. Thecontextual parameter determiner 820 receives thecurrent location 720 and, for example, separates out the location and time information and sends the information to the self-awareness performance parameter identifier 830 so that it can use the information to identify precise time and location-specific performance parameters.

采用关于车辆位置和当前时间的信息，自身觉知性能参数更新器830可访问内在性能模型840和/或外在性能模型850，以取得当前位置和时间特有的性能相关参数值。在某些实施例中，内在性能模块840可被配置为具体指定与车辆内在性能有关的参数的类型及其当前的值。类似地，外在性能模型850可被配置为具体指定对车辆运行能力有影响的参数的类型及其当前的值。Using the information about the vehicle's position and current time, the self-awareness performance parameter updater 830 may access the intrinsic performance model 840 and/or theextrinsic performance model 850 to obtain performance-related parameter values specific to the current position and time. In some embodiments, the intrinsic performance module 840 may be configured to specify the types of parameters related to the intrinsic performance of the vehicle and their current values. Similarly, theextrinsic performance model 850 may be configured to specify the types of parameters that affect vehicle performance and their current values.

在运行中，为了将参数的值保持为当前的值，内在和外在性能模型(840和850)可规则地触发更新器(860-a，……，860-f)，以收集实时信息并基于这样收集的实时信息来更新对应参数的值。例如，内在性能模型840可以被配置为具有这样的机制：其致动车辆性能参数更新器860-a，以收集与车辆的内在性能有关的更新信息。这样的机制可具体指定不同模式的触发。例如，其可以基于规则的计划，例如，每天或每小时。其还可以具体指定将由某外部事件触发，例如从维修店或从车上的检测车辆部件的某些功能状态已经变化的传感器接收的信号。在这种情况下，车辆性能参数更新器860-a可接收来自传感器的实时车辆信息，并更新内在性能模型中的相关性能参数的值/状态，以反映车辆的实时状态。例如，在车辆运行过程中，前灯或制动器可能变得失去功能。这样的实时检测到的信息可由车辆性能参数更新器860-a收集，并用于更新存储在内在性能参数存储器840中的信息。这样的与车辆有关的更新信息于是可由自身觉知性能参数生成器840用于生成内在性能参数。In operation, in order to maintain the value of the parameter at the current value, the intrinsic and extrinsic performance models (840 and 850) can regularly trigger the updater (860-a, . . . , 860-f) to collect real-time information and The values of the corresponding parameters are updated based on the real-time information thus collected. For example, intrinsic performance model 840 may be configured with a mechanism that actuates vehicle performance parameter updater 860-a to collect updated information related to the intrinsic performance of the vehicle. Such a mechanism may specify different modes of triggering. For example, it can be based on a rule-based schedule, eg, daily or hourly. It may also specify that it will be triggered by some external event, such as a signal received from a repair shop or from a sensor on the vehicle that detects that certain functional states of vehicle components have changed. In this case, the vehicle performance parameter updater 860-a may receive real-time vehicle information from sensors and update the values/states of the relevant performance parameters in the intrinsic performance model to reflect the real-time state of the vehicle. For example, headlights or brakes may become inoperative during vehicle operation. Such real-time detected information may be collected by the vehicle performance parameter updater 860 - a and used to update the information stored in the intrinsic performance parameter memory 840 . Such vehicle-related updated information can then be used by the self-aware performance parameter generator 840 to generate intrinsic performance parameters.

类似地，外在性能模型850可被配置为具体指定用于更新不同类型的外在性能参数的更新机制。更新机制可具体指定按规律计划的更新，或者由某些事件触发的更新。不同类型的外在性能参数可被配置为基于不同的触发机制来更新。例如，对于天气相关外在性能参数或者是可能与天气密切关联的外在性能参数(例如车辆附近的可见度)，更新可以规律地进行，例如，每几分钟。类似地，交通敏感参数——例如常常作为交通状况的直接结果的实际可允许速度——也可有规律地更新。不同类型的参数尽管全都有规律地更新，可具有不同的更新计划，这些计划可能在从每隔几秒到每隔几分钟或每隔几小时的范围内。Similarly,extrinsic performance model 850 may be configured to specify update mechanisms for updating different types of extrinsic performance parameters. The update mechanism can specify updates that are scheduled on a regular basis, or that are triggered by certain events. Different types of extrinsic performance parameters can be configured to be updated based on different triggering mechanisms. For example, for weather-related extrinsic performance parameters, or extrinsic performance parameters that may be closely related to weather (eg, visibility in the vicinity of the vehicle), the update may occur regularly, eg, every few minutes. Similarly, traffic-sensitive parameters, such as actual allowable speeds, often as a direct result of traffic conditions, may also be updated regularly. Different types of parameters, although all updated regularly, may have different update schedules, which may range from every few seconds to every few minutes or every few hours.

另一方面，某些外在性能相关参数可在某些事件发生时做出。例如，对于方位敏感参数(例如，太阳眩光是否存在)，更新可以在车辆以特定方向行进时触发。如果车辆行进的方向在某个下午时间从北变为西北，这可触发方位敏感参数更新器860-d，以收集与太阳眩光有关的信息，并更新关于太阳眩光的情况。在某些情况下，更新可指示不存在太阳眩光，例如，当天气是多云天时。在某些情况下，更新可指示存在太阳眩光。在任一情况下，这种方位敏感信息于是用于更新存储在外在性能参数存储器850中的对应的外在性能参数的值。类似地，时间敏感参数(例如由于一天中的时间的车辆可视度)的更新可基于检测到的位置、位置的时区以及该时刻在一天中的具体时间而触发。在某些实施例中，某些性能参数的更新也可由与检测到的其他性能参数值的更新有关的事件触发。例如，例如湿滑道路状况的道路敏感参数更新可在天气状况更新指示天气开始下雨或下雪时触发。On the other hand, certain extrinsic performance-related parameters can be made when certain events occur. For example, for orientation-sensitive parameters (eg, the presence or absence of sun glare), an update can be triggered when the vehicle is traveling in a particular direction. If the direction of travel of the vehicle changes from north to northwest at some afternoon time, this may trigger the orientation sensitive parameter updater 860-d to collect information related to sun glare and update the situation regarding sun glare. In some cases, the update may indicate that there is no sun glare, for example, when the weather is cloudy. In some cases, the update can indicate the presence of sun glare. In either case, this orientation-sensitive information is then used to update the value of the corresponding extrinsic performance parameter stored in the extrinsicperformance parameter store 850 . Similarly, updates to time-sensitive parameters (eg, vehicle visibility due to time of day) may be triggered based on the detected location, the time zone of the location, and the specific time of day at that moment. In some embodiments, updates of certain performance parameters may also be triggered by events related to detected updates of other performance parameter values. For example, road sensitive parameter updates such as slippery road conditions may be triggered when a weather condition update indicates that the weather is starting to rain or snow.

在所示的实施例中，车辆性能参数更新器860-a接收来自存储器750的静态车辆信息和来自实时车辆信息馈送(其可能来自多种来源)的动态车辆信息更新。这些来源的实例包括经销商、车辆维护场所、报告部件状态变化的车上传感器或其他来源。天气敏感参数更新器860-b可接收动态天气更新以及其他天气敏感性能参数(例如降水、可见度、雾或与天气有关且潜在影响车辆运行的任何其他参数)的更新。天气相关信息可来自馈送实时数据的多种数据源。In the illustrated embodiment, vehicle performance parameter updater 860-a receives static vehicle information frommemory 750 and dynamic vehicle information updates from a real-time vehicle information feed (which may come from a variety of sources). Examples of these sources include dealerships, vehicle maintenance sites, on-board sensors reporting changes in component status, or other sources. Weather-sensitive parameter updater 860-b may receive dynamic weather updates as well as updates to other weather-sensitive performance parameters such as precipitation, visibility, fog, or any other parameter that is weather-related and potentially affects vehicle operation. Weather-related information can come from a variety of data sources that feed real-time data.

交通敏感参数更新器860-c可接收动态交通报告和与可能影响车辆运行的交通有关的其他信息。实例包括交通堵塞的程度(其可用于确定车辆路径是否需要重新规划)或已经导致交通拥挤的事件的时间(以推定延迟将会持续多久，从而判断是否重新作出路径规划)。交通或交通相关信息可从一个或多于一个的用于实时数据馈送的源接收。方位敏感参数更新器860-d可被配置为收集车辆方向上沿着道路的信息。这种方位敏感信息可包括特定方向(例如东或者西)的太阳眩光，或车辆位于的道路方向上的任何潜在情况(例如道路前方的滑坡情况)。类似地，一旦触发，道路敏感参数更新器860-e可以关于车辆位置从一个或多于一个实时信息馈送源收集关于多种道路或道路状况的信息。这样的信息可以与道路(例如开通、封闭、绕行、学区等)或其状况(例如湿滑、结冰、淹水、施工等)有关。时间敏感参数更新器860-f可被配置为从数据源实时收集依赖于时间的实时数据。例如，道路的可见度可依赖于车辆所位于的时区中一天当中的时间。The traffic sensitive parameter updater 860-c may receive dynamic traffic reports and other information related to traffic that may affect vehicle operation. Examples include the degree of traffic congestion (which can be used to determine whether vehicle routing needs to be re-routed) or the time of the event that has caused the traffic congestion (to estimate how long the delay will last to determine whether to re-route). Traffic or traffic-related information may be received from one or more sources for real-time data feeds. The orientation sensitive parameter updater 860-d may be configured to collect information along the road in the direction of the vehicle. Such orientation sensitive information may include sun glare in a particular direction (eg east or west), or any potential situation in the direction of the road in which the vehicle is located (eg a landslide situation ahead of the road). Similarly, once triggered, the road sensitive parameter updater 860-e may collect information about various roads or road conditions from one or more real-time information feeds with respect to vehicle location. Such information may relate to a road (eg, open, closed, detour, school zone, etc.) or its condition (eg, slippery, icy, flooded, construction, etc.). The time-sensitive parameter updater 860-f may be configured to collect time-dependent real-time data from data sources in real-time. For example, the visibility of the road may depend on the time of day in the time zone in which the vehicle is located.

所收集的实时数据于是可用于更新内在性能模型840和/或外在性能模型850。可对这样的更新数据加时间戳。自身觉知性能参数更新器830于是可访问内在和外在性能模型840和850二者，以确定相关的性能参数及其更新值。所取得的内在/外在性能参数于是可被输出，使得它们能由图5所示的多种规划模块使用。具体而言，这样生成的自身觉知性能参数510由路径规划模块550用于路径规划，如将参照图10-13所讨论的那样。自身觉知性能参数也由运动规划模块560用于个性化运动规划，其将在下面参照图14-19公开。自身觉知性能参数也由车道规划模块570用于车道控制，其将参照图20-22详细介绍。The collected real-time data can then be used to update the intrinsic performance model 840 and/or theextrinsic performance model 850 . Such update data may be time stamped. Self-aware performance parameter updater 830 can then access both intrinsic andextrinsic performance models 840 and 850 to determine relevant performance parameters and their updated values. The acquired intrinsic/extrinsic performance parameters can then be exported so that they can be used by the various planning modules shown in FIG. 5 . In particular, the self-awareness performance parameters 510 thus generated are used by thepath planning module 550 for path planning, as will be discussed with reference to Figures 10-13. Self-awareness performance parameters are also used by themotion planning module 560 for personalized motion planning, which will be disclosed below with reference to Figures 14-19. The self-awareness performance parameter is also used for lane control by thelane planning module 570, which will be described in detail with reference to FIGS. 20-22.

图9为根据本示教一实施例的自身觉知性能参数生成器740的示例性过程的流程图。首先，在910处接收当地背景信息，在920处提取位置和时间信息，其在930处由不同的更新器用于获得来自与内在和外在性能的多种方面有关的不同源的信息馈送。这样获得的信息于是由不同的更新器在940处使用，以更新内在性能参数840和外在性能参数850。基于当前位置、时间和接收的当地背景信息，自身觉知性能参数更新器830于是识别当前时刻与车辆有关的多种内在和外在性能参数510，以便在940处更新内在/外在性能参数，并在950处生成更新的性能参数。这样更新的内在/外在性能参数510于是在960处输出。FIG. 9 is a flowchart of an exemplary process of self-awarenessperformance parameter generator 740 according to an embodiment of the present teachings. First, local context information is received at 910, location and time information is extracted at 920, which is used at 930 by different updaters to obtain information feeds from different sources related to various aspects of intrinsic and extrinsic performance. The information thus obtained is then used at 940 by the different updaters to update the intrinsic 840 andextrinsic performance parameters 850 . Based on the current location, time, and received local context information, the self-awareness performance parameter updater 830 then identifies various intrinsic andextrinsic performance parameters 510 associated with the vehicle at the current moment to update the intrinsic/extrinsic performance parameters at 940, And generate updated performance parameters at 950 . Such updated intrinsic/extrinsic performance parameters 510 are then output at 960 .

这样动态收集的自身觉知性能参数将被用在多种车辆行为规划操作中，包括路径规划、运动规划以及车道相关车辆行为规划。例如，在人类驾驶中，选择到目的地的路径常常在考虑由自身觉知性能参数捕获的因素的情况下做出。例如，人类驾驶者可以基于例如车辆装有什么或能做什么(内在性能参数)来选择到希望的目的地的路径。如果车辆处于不能良好处理陡峭道路的状况，则需要避开这样的道路。另外，人类驾驶者也可考虑其他因素，例如当天的天气，所考虑道路的状况，为一天中的特定时间所计划的或已知的事件(外在性能参数)。例如，如果一条道路指向西方且当时太阳将要下山，可能将有太多眩光，故最好是走另一条替代道路。为了安全和可靠的目的，自动车辆也应在路径规划期间就路径选择考虑这些内在和外在性能。Such dynamically collected self-awareness performance parameters will be used in a variety of vehicle behavior planning operations, including path planning, motion planning, and lane-related vehicle behavior planning. For example, in human driving, choosing a route to a destination is often made taking into account factors captured by self-awareness performance parameters. For example, a human driver may select a route to a desired destination based on, for example, what the vehicle is equipped with or can do (intrinsic performance parameters). If the vehicle is in a condition that does not handle steep roads well, it is necessary to avoid such roads. In addition, the human driver may also take into account other factors, such as the weather of the day, the conditions of the road under consideration, planned or known events (extrinsic performance parameters) for a particular time of day. For example, if a road is pointing west and the sun is about to go down, there may be too much glare, so it is better to take an alternative road. For safety and reliability purposes, autonomous vehicles should also consider these intrinsic and extrinsic properties for path selection during path planning.

传统路径规划方法常采用某种成本函数(cost function)，使得所选路径的成本最小化。例如，传统路径规划考虑例如行驶距离最优化、抵达目的地所需时间最小化或是使到达目的地使用的燃料最小化。在某些实例中，在最优化成本时，传统方法也可考虑交通状况，例如，高交通量路径可减小速度，导致到达目的地的增多的时间和燃料。这些最优化功能常常假设所有车辆能以同样的方式处理所有路径，且所有路径能被同样好地处理。这样的假设常常并不是真的，故当自动驾驶车辆应用这样的规划方案时，它们常常发现不能进行或甚至是在某些情况下变得很危险。本示教的目的在于实现安全、实际、可靠、适应于变化的与内在和外在性能有关的参数的路径规划。Traditional path planning methods often use a certain cost function to minimize the cost of the selected path. For example, traditional route planning considers, for example, optimizing the distance traveled, minimizing the time required to reach the destination, or minimizing the fuel used to reach the destination. In some instances, conventional approaches may also take into account traffic conditions when optimizing costs, eg, high-traffic paths may reduce speed, resulting in increased time and fuel to the destination. These optimization functions often assume that all vehicles can handle all paths in the same way, and that all paths can be handled equally well. Such assumptions are often not true, so when autonomous vehicles apply such planning schemes, they often find it impossible or even dangerous in some cases. The purpose of this teaching is to achieve safe, practical, reliable, and adaptable path planning for changing parameters related to intrinsic and extrinsic performance.

如图5所示，规划模块450在实现不同的规划任务(包括路径规划模块550、运动规划模块560和车道规划模块570)时考虑自身觉知性能参数510。下面，参照图10-13，提供了关于路径规划模块550的细节。图10示出了根据本示教一实施例的路径规划模块550的示例性高层次系统图。路径规划模块550的目的是，在内在和外在性能方面，均以自身觉知的方式，基于希望的目的地来规划路径。相反，传统的路径规划技术主要考虑例如最短距离、最短时间、最大利用高速公路/本地道路等标准，而不考虑动态内在性能参数和实时外在性能参数。As shown in FIG. 5, theplanning module 450 considers the self-awareness performance parameter 510 when implementing different planning tasks, including thepath planning module 550, themotion planning module 560, and thelane planning module 570. Below, with reference to Figures 10-13, details regarding thepath planning module 550 are provided. FIG. 10 shows an exemplary high-level system diagram of apath planning module 550 in accordance with an embodiment of the present teachings. The purpose of theroute planning module 550 is to plan a route based on a desired destination, both in terms of intrinsic and extrinsic performance, in a self-aware manner. On the contrary, traditional path planning techniques mainly consider criteria such as shortest distance, shortest time, maximum utilization of expressways/local roads, etc., without considering dynamic intrinsic performance parameters and real-time extrinsic performance parameters.

在此所示实施例中，路径规划模块550包括路径选择偏好判断器1030和全局路径规划器1020。路径选择偏好判断器1030判断在选择路径时将要考虑的偏好。全局路径规划器1020基于包括自身觉知性能参数150在内的多种信息来选择适当的路径。在某些实施例中，路径规划活动可基于所示的路径规划触发信号而被触发。在被致动时，全局路径规划器1020可收集与当前路径规划操作有关的多种类型的动态信息。例如，全局路径规划器1020可依赖于与原点/当前位置有关的信息和希望的目的地。规划是关于原点/当前位置以及目的地来执行的。目的地信息可以用不同的方式来确定。例如，其可视情况可选地经由接口单元1010从驾驶者/乘员接收。In the embodiment shown here, thepath planning module 550 includes a pathselection preference determiner 1030 and aglobal path planner 1020 . The routeselection preference judger 1030 judges the preference to be considered when selecting a route. Theglobal path planner 1020 selects an appropriate path based on various information including self-awareness performance parameters 150 . In some embodiments, the path planning activity may be triggered based on the path planning trigger signal shown. When activated, theglobal path planner 1020 may collect various types of dynamic information related to current path planning operations. For example, theglobal path planner 1020 may rely on information about the origin/current location and the desired destination. Planning is performed with respect to the origin/current position as well as the destination. Destination information can be determined in different ways. For example, it may optionally be received from the driver/occupant via theinterface unit 1010 as appropriate.

全局路径规划器1020也可将实时数据480用作输入，并相应地规划路径。如参照图4B所讨论的，实时数据包括与实时车辆相关信息(位置)有关的信息、关于观察到的车辆内的乘员的信息、……、以及道路特性。这样的实时数据提供路径规划需要的周围环境信息。全局路径规划器1020也接收自身觉知性能参数510，其向规划器通知，在规划的时候的给定动态内在以及外在情况的条件下，什么是可能的。例如，内在性能参数可指示车辆当前由于某些机械问题而不能快速行驶，故全局路径规划器1020可将之考虑在内，以便规划例如主要涉及本地道路并可能经过某些汽车维修店的路径。类似地，外在性能参数可指示在车辆当前位置的北方，太阳眩光相当强烈，故全局路径规划器可基于该信息以在太阳下山之前避开位于北方的附近的路径。实时数据480和自身觉知性能参数510向全局路径规划器1020提供信息，以便使它能够规划出在给定例如当前时间、当前车辆位置、当前天气、当前乘员情况和当前道路状况的条件下适当的路径。Theglobal path planner 1020 may also use the real-time data 480 as input and plan the path accordingly. As discussed with reference to FIG. 4B , the real-time data includes information related to real-time vehicle-related information (location), information about observed occupants within the vehicle, . . . , and road characteristics. Such real-time data provides surrounding environment information needed for route planning. Theglobal path planner 1020 also receives self-awareness performance parameters 510, which inform the planner what is possible given the dynamic intrinsic and extrinsic conditions at the time of planning. For example, intrinsic performance parameters may indicate that the vehicle is currently unable to travel fast due to some mechanical problem, so theglobal path planner 1020 may take this into account in order to plan a path that involves mostly local roads and possibly some auto repair shops, for example. Similarly, the extrinsic performance parameter may indicate that the sun glare is fairly strong to the north of the vehicle's current location, so the global path planner may use this information to avoid paths that lie near the north until the sun goes down. The real-time data 480 and self-awareness performance parameters 510 provide information to theglobal path planner 1020 so that it can plan the appropriate path given conditions such as current time, current vehicle location, current weather, current occupant conditions, and current road conditions. path of.

全局路径规划器1020也可考虑将在路径规划中应用的偏好。这样的偏好可以由驾驶者/乘员经由用户接口单元1010具体指定(其可被传送到全局路径规划器1020)，或者，可经由其他装置获得(见下面参照图12的公开)。在考虑将要应用的偏好时，存储在路径选择偏好配置1050中的信息也可被访问和考虑。这样的路径选择偏好配置可具体指定不同场景下路径选择中的某些一般性的偏好，例如，在下雨/下雪场景下避免陡峭/弯曲道路，在夜间避开小道，避开加油站非常少的道路，等等。全局路径规划器1020可将从实时数据480接收的相关信息和自身觉知性能参数510传送给路径选择偏好判断器1030，其又被路径选择判断器1030用于从1050取得特定的路径选择偏好配置。例如，如果现在正在下雪(由实时数据480)，且车辆没有雪胎(由内在性能参数510)，这样的动态信息可从全局路径规划器1020转送给路径选择偏好判断器1030，使得与这样的动态场景有关的选择偏好配置可从路径选择偏好配置1050取得(例如避开陡峭/弯曲道路)，并被送回到全局路径规划器1020，使得它能在选择合适的路径时被依赖。Theglobal path planner 1020 may also consider preferences to be applied in path planning. Such preferences may be specified by the driver/occupant via the user interface unit 1010 (which may be communicated to the global path planner 1020), or may be obtained via other means (see disclosure with reference to Figure 12 below). Information stored in routingpreference configuration 1050 may also be accessed and taken into account when considering the preferences to be applied. Such a routing preference configuration can specify some general preferences in routing in different scenarios, for example, avoid steep/curvy roads in rain/snow scenarios, avoid trails at night, and avoid very few gas stations road, etc. Theglobal path planner 1020 may pass the relevant information received from the real-time data 480 and the self-aware performance parameters 510 to the pathselection preference determiner 1030 , which in turn is used by thepath selection determiner 1030 to obtain the particular routing preference configuration from 1050 . . For example, if it is currently snowing (by real-time data 480 ) and the vehicle has no snow tires (by intrinsic performance parameters 510 ), such dynamic information may be forwarded fromglobal path planner 1020 to pathselection preference determiner 1030 such that The dynamic scene-related selection preference configuration of 1050 can be taken from the path selection preference configuration 1050 (eg avoiding steep/curvy roads) and sent back to theglobal path planner 1020 so that it can be relied upon in choosing an appropriate path.

为了确定合适的路径，除了知道选择偏好以外，全局路径规划器1020也可能需要知道关于道路的附加的信息，例如从车辆当前位置到预定的目的地有什么路径可用。另外，对于各个可用的路径，地图/道路配置1060也可存储关于各个可用道路/路径的特性信息。道路/路径的这样的特性信息可包括、但不限于几何特性，例如道路/路径的性质(高速公路或者非高速公路)、道路/路径的维度、陡峭度/弯曲度、道路/路径的状况，等等。在规划时，全局路径规划器1020可首先确定车辆当前位置到希望的目的地之间的可用道路/路径。为了选择到目的地的适当的路径，对于这样的可用道路/路径，它们的特性信息也可由全局路径规划器1020访问，使得选择可基于这样的特性信息做出。In order to determine an appropriate route, in addition to knowing the selection preferences, theglobal route planner 1020 may also need to know additional information about the road, such as what routes are available from the vehicle's current location to a predetermined destination. Additionally, for each of the available paths, the map/road configuration 1060 may also store characteristic information about each of the available roads/paths. Such characteristic information of the road/path may include, but is not limited to, geometric characteristics such as the nature of the road/path (freeway or non-highway), dimensions of the road/path, steepness/curvature, condition of the road/path, and many more. In planning, theglobal path planner 1020 may first determine the available roads/paths between the vehicle's current location and the desired destination. In order to select an appropriate route to a destination, for such available roads/paths, their characteristic information may also be accessed by theglobal route planner 1020, so that selections can be made based on such characteristic information.

采用关于可用道路/路径的信息以及关于这些可用道路/路径的特性信息，全局路径规划器1020于是可通过将由路径选择偏好判断器1030确定的路径选择偏好与可用道路/路径的特性信息相匹配来选择到目的地的适当的路径。关于全局路径规划器1020的细节参照图12-13提供。Using the information about the available roads/paths and the characteristic information about these available roads/paths, theglobal path planner 1020 can then match the path selection preferences determined by the pathselection preference determiner 1030 with the characteristic information of the available roads/paths Choose the appropriate path to the destination. Details regarding theglobal path planner 1020 are provided with reference to Figures 12-13.

如前面所讨论的，全局路径规划器1020基于来自不同来源(包括实时数据480和自身觉知性能参数510)的动态信息来选择规划的路径。除此之外，由于车辆可能在移动中，或目的地可能随时间变化，车辆当前位置和目的地可能随时间改变，正如实时数据480和自身觉知性能参数510一样。当这样的信息改变时，其可以影响所规划的全局路径。例如，在当前位置变化时，与当前位置相关联的实时数据也可变化，例如，从与前一地点相关联的好天气变为与当前位置关联的下雨的状况。这又可能导致路径选择偏好方面的变化，最终，导致所选择路径的变化。因此，全局路径规划器1020可以以双向方式和动态方式与路径选择偏好判断器1030交互。每当可能使得有必要重新确定路径选择偏好的变化存在时，全局路径规划器1020于是可致动路径选择偏好判断器1030，以便修改或重新生成将由全局路径规划器1020用于在给定情况下确定合适路径的偏好。As previously discussed, theglobal path planner 1020 selects a planned path based on dynamic information from various sources, including real-time data 480 and self-awareness performance parameters 510 . Additionally, as the vehicle may be in motion, or the destination may change over time, the vehicle's current location and destination may change over time, as does the real-time data 480 and self-awareness performance parameters 510 . When such information changes, it can affect the planned global path. For example, as the current location changes, the real-time data associated with the current location may also change, eg, from good weather associated with the previous location to rainy conditions associated with the current location. This in turn may lead to changes in path selection preferences and, ultimately, changes in the path chosen. Thus, theglobal path planner 1020 can interact with the pathselection preference determiner 1030 in a bidirectional and dynamic manner. Theglobal path planner 1020 may then actuate the pathselection preference determiner 1030 whenever a change exists that may make it necessary to re-determine the path selection preferences, so that the modification or regeneration will be used by theglobal path planner 1020 in a given situation Determine preferences for suitable paths.

图11为根据本示教一实施例用于路径规划模块550的示例性过程的流程图。关于车辆目的地的信息以及视情况可选的关于偏好的信息在1110处被接收。实时数据480以及自身觉知性能参数510由全局路径规划器1020在1120处接收，与车辆的当前场景或情况有关的多种信息于是可在1130处从接收的实时数据和自身觉知性能参数识别。基于与当前场景有关的相关信息，在1140处确定对于当前场景特定的偏好。为了规划路径，全局路径规划器1020在1150处访问关于基于当前位置和希望目的地的可用道路/路径的信息以及这种可用路径/路径的特性信息。在1160处，基于在当前场景的基础上确定的特定偏好以及道路/场景信息，全局路径规划器1020选择对于当前情况合适的路径。11 is a flowchart of an exemplary process forpath planning module 550 in accordance with an embodiment of the present teachings. Information about the vehicle's destination and, optionally, preferences is received at 1110 . The real-time data 480 and the self-awareness performance parameters 510 are received by theglobal path planner 1020 at 1120, and various information related to the current scene or situation of the vehicle can then be identified from the received real-time data and self-awareness performance parameters at 1130 . Based on the relevant information about the current scene, preferences specific to the current scene are determined at 1140 . To plan a route, theglobal route planner 1020 accesses, at 1150, information about available roads/paths based on the current location and desired destination, and information on characteristics of such available routes/paths. At 1160, theglobal path planner 1020 selects a suitable path for the current situation based on the specific preferences and road/scenario information determined on the basis of the current scene.

图12示出了根据本示教一实施例的全局路径规划器1020的示例性高层次系统图。在此所示实施例中，全局路径规划器1020包含自身觉知性能参数分析器1205、基于内在性能的过滤器生成器1210以及路径选择引擎1230。视情况可选地，全局路径规划器1020也包含目的地更新器1225，用于动态确定和更新当前目的地。在所示的实施例中，全局路径规划器1020也视情况可选地包括用于对驾驶者/乘员偏好进行个性化的机制，使得在选择路径时使用。路径选择偏好判断器1030将基于车辆当前处于的特定情况确定与选择路径有关的偏好(其不同于获得针对特定驾驶者/乘员的个性化偏好)。Figure 12 shows an exemplary high-level system diagram of aglobal path planner 1020 in accordance with an embodiment of the present teachings. In this illustrated embodiment, theglobal path planner 1020 includes a self-awareperformance parameter analyzer 1205, an intrinsic performance-basedfilter generator 1210, and apath selection engine 1230. Optionally, theglobal path planner 1020 also includes adestination updater 1225 for dynamically determining and updating the current destination. In the illustrated embodiment, theglobal path planner 1020 also optionally includes a mechanism for personalizing driver/occupant preferences for use when selecting a path. The routeselection preference determiner 1030 will determine preferences related to choosing a route based on the particular situation the vehicle is currently in (as opposed to obtaining personalized preferences for a particular driver/occupant).

如说明性地示出的，视情况可选的确定个性化偏好的机制包含乘员驾驶数据分析器1245、偏好个性化模块1250和乘员偏好判断器1240。在运行中，乘员驾驶数据分析器1245接收作为输入的所记录的人类驾驶数据430，并从这种数据中分析或学习，以理解与特定的驾驶者/乘员有关的路径偏好。例如，由记录的人类驾驶数据430，可以习得特定驾驶者偏爱在本地道路上而不是在高速公路上行驶，或是历史上选择在夜晚使用高速公路、即使这涉及长得多的距离。还可以学习与车辆相关联的所有驾驶者的偏好。例如，多个人(一家的丈夫、妻子和孩子)可以与车辆相关联，即，这些人中的任何一个可以操作车辆。乘员驾驶数据分析器1245可从记录的人类驾驶数据430中学习与这类驾驶者的驾驶行为相关联的多种类型的信息，这可使得偏好个性化模块1250能够在接收到这种驾驶行为信息时建立每一个这类个体的个人偏好。As illustratively shown, an optional mechanism for determining personalized preferences includes an occupant drivingdata analyzer 1245 , apreference personalization module 1250 , and anoccupant preference determiner 1240 . In operation, the occupant drivingdata analyzer 1245 receives as input the recordedhuman driving data 430 and analyzes or learns from such data to understand route preferences associated with a particular driver/occupant. For example, from recordedhuman driving data 430, it may be learned that a particular driver prefers to drive on local roads over highways, or has historically chosen to use highways at night, even though this involves much longer distances. The preferences of all drivers associated with the vehicle can also be learned. For example, multiple persons (a husband, wife, and children of a family) may be associated with the vehicle, ie, any one of these persons may operate the vehicle. The occupant drivingdata analyzer 1245 may learn various types of information associated with the driving behavior of such drivers from the recordedhuman driving data 430, which may enable thepreference personalization module 1250 to receive such driving behavior information to establish the individual preferences of each such individual.

在从乘员驾驶数据分析器1245接收到与各个个体驾驶者有关的信息时，偏好个性化模块1250于是可生成路径选择方面的个性化偏好。这样的路径相关偏好可以不仅反映路径选择，还代表不同情况(例如一天当中的特定时间帧、季节、位置等)下的路径选择偏好。这样对于各个个体驾驶者建立的偏好于是可存储在存储器1265中。在路径规划时，乘员偏好判断器1240接收实时数据480，并且，基于实时数据480中的多种信息(例如月/日/时间、乘员信息、区域天气等)，乘员偏好判断器1240可从路径选择偏好存储器1265访问能在当前路径规划中应用的相关偏好。例如，如果实时数据指示驾驶者是特定的人，且当前时间是1月的下午7：45等，乘员偏好判断器1240可在1265处与当前特定驾驶者有关地识别个性化的路径偏好，其与一年中的季节以及特定时间帧相关(例如，驾驶者可能偏爱在冬季在高速公路上行驶)。如此识别的个性化路径选择偏好于是可发送到路径选择引擎1230，使得路径规划时的驾驶者/乘员的个性化偏好能在确定选择哪一路径时被考虑在内。Upon receiving information from occupant drivingdata analyzer 1245 relating to various individual drivers,preference personalization module 1250 may then generate personalized preferences in routing. Such route-related preferences may not only reflect route choice, but also represent route choice preferences under different circumstances (eg, specific time frames of the day, seasons, locations, etc.). The preferences thus established for each individual driver may then be stored inmemory 1265 . During route planning, theoccupant preference determiner 1240 receives real-time data 480, and based on various information in the real-time data 480 (eg, month/day/time, occupant information, regional weather, etc.), theoccupant preference determiner 1240 can retrieve information from the route Theselection preference store 1265 accesses relevant preferences that can be applied in the current path plan. For example, if the real-time data indicates that the driver is a particular person, and the current time is 7:45 pm in January, etc., theoccupant preference determiner 1240 may, at 1265, identify personalized route preferences in relation to the current particular driver, which Depends on the season of the year and a specific time frame (eg, a driver may prefer to drive on the highway in winter). The personalized routing preferences so identified may then be sent to therouting engine 1230 so that the driver/occupant's personalized preferences in routing can be taken into account in determining which routing to choose.

如图12所示，路径选择引擎1230也可将由路径选择偏好判断器1030推定的偏好用作输入，并将之用于其路径选择操作。在某些实施例中，路径选择引擎1230可依赖于来自1030的偏好，而不考虑驾驶者的个性化偏好，即，它可在其路径选择中仅仅依赖于由路径选择偏好判断器1030识别的偏好。As shown in FIG. 12, therouting engine 1230 may also use the preferences inferred by therouting preference determiner 1030 as input and use them for its routing operations. In some embodiments, therouting engine 1230 may rely on preferences from 1030 without regard to the driver's personalized preferences, ie, it may rely solely on the preferences identified by therouting preference determiner 1030 in its routing preference.

在选择适合用于当前情况的路径时，路径选择引擎1230也可接收自身觉知性能参数510。在所示的实施例中，自身觉知性能参数分析器1205分离外在性能参数和内在性能参数，并将外在性能参数发送到路径选择引擎1230，使得与车辆所处当前情况相关联的外在状况能在路径选择中被考虑在内。例如，外在性能参数可指示路径7上存在正在进行的施工，路径选择引擎1230可考虑这一点并避开路径7。然而，如果目的地当前被设置为路径7上的学校，且驾驶者的习惯是每天在当前时间(例如下午3：30)从学校接孩子，在考虑所有事情的情况下，路径选择引擎1230可选择选用路径7。Path selection engine 1230 may also receive self-awareness performance parameters 510 when selecting a path appropriate for the current situation. In the illustrated embodiment, the self-awarenessperformance parameter analyzer 1205 separates the extrinsic performance parameters from the intrinsic performance parameters and sends the extrinsic performance parameters to therouting engine 1230 so that the extrinsic performance parameters associated with the current situation the vehicle is in are The situation can be taken into account in routing. For example, an extrinsic performance parameter may indicate that there is ongoing construction on path 7, which therouting engine 1230 may take into account and avoid path 7. However, if the destination is currently set to school on route 7, and the driver's habit is to pick up the child from school at the current time (eg, 3:30 pm) every day, all things considered, therouting engine 1230 may Choose to use path 7.

类似地，内在性能参数也可在选择适当的路径时被考虑在内。在此所示的实施例中，内在性能参数被馈送到基于内在性能的过滤器生成器1210，其可基于内在性能参数产生不同的过滤器1215，使得这样的过滤器可由路径选择引擎用于滤除在给定内在性能参数的情况下不合适的路径。例如，如果内在性能参数指示车辆没有雪胎，任何陡峭和/或弯曲的路径可能在雪天不合适。Similarly, intrinsic performance parameters can also be taken into account when selecting an appropriate path. In the embodiment shown here, the intrinsic performance parameters are fed to an intrinsic performance basedfilter generator 1210, which can generatedifferent filters 1215 based on the intrinsic performance parameters, such that such filters can be used by the routing engine to filter Except for paths that are not suitable given intrinsic performance parameters. For example, if intrinsic performance parameters indicate that the vehicle does not have snow tires, any steep and/or curved trails may not be suitable in snow.

基于由当前位置更新器1235跟踪的车辆当前位置和由目的地更新器1225跟踪的目的地二者，路径选择引擎1230选择路径。有些情况下，改变的当前位置和目的地可触发路径选择引擎1230，以致动路径选择偏好判断器1030，以便在给定变化情况下重新评估路径选择中的偏好。Routing engine 1230 selects a route based on both the vehicle's current location tracked bycurrent location updater 1235 and the destination tracked bydestination updater 1225 . In some cases, the changed current location and destination may trigger therouting engine 1230 to actuate therouting preference determiner 1030 to re-evaluate preferences in routing given the change.

图13为根据本示教一实施例的全局路径规划器1020的示例性过程的流程图。在1310处，自身觉知性能参数被接收。内在性能参数用于在1320处生成基于内在性能的过滤器，使得特定路径能在给定车辆内在状况的情况下由于不适合而被滤除。外在性能参数在1330处从接收的自身觉知性能参数提取。同时，实时数据480在1340处被连续接收，且记录的人类驾驶数据在1350处被接收。这样的数据于是用于在1360处确定与当前驾驶者、当前情况、当前时间有关的个性化路径选择偏好。在1370处，自身觉知性能参数和/或驾驶者个性化偏好于是可以用于在考虑所有因素的情况下选择合适的路径。在1380处，输出所选择的路径。13 is a flowchart of an exemplary process of theglobal path planner 1020 according to an embodiment of the present teachings. At 1310, self-awareness performance parameters are received. The intrinsic performance parameters are used to generate intrinsic performance based filters at 1320 so that certain paths can be filtered out as unsuitable given the vehicle intrinsic conditions. Extrinsic performance parameters are extracted at 1330 from the received self-aware performance parameters. Meanwhile, real-time data 480 is continuously received at 1340 and recorded human driving data is received at 1350 . Such data is then used at 1360 to determine personalized routing preferences related to the current driver, current situation, current time. At 1370, self-awareness performance parameters and/or driver personalization preferences may then be used to select an appropriate route taking all factors into consideration. At 1380, the selected path is output.

根据本示教的路径规划允许在路径规划中考虑多种类型的信息，例如实时数据和自身觉知性能参数，使得规划的路径适应当时的车辆状况(经由内在性能参数)、当时车辆所处的动态环境(经由动态数据以及外在性能参数)、基于例如动态更新实时数据(见图4B)确定的乘员特性以及乘员个性化偏好。类似地，这样的信息也可在其他类型的规划操作中使用，使得规划的车辆活动适应于实时情况，基于个体偏好得到个性化，并且允许车辆的行为更像人类驾驶者。下面，参照关于个性化适应性运动规划的图14-19提供更多的细节。Path planning according to the present teachings allows multiple types of information, such as real-time data and self-aware performance parameters, to be considered in the path planning, so that the planned path is adapted to the current vehicle conditions (via intrinsic performance parameters), the vehicle's location at that time Dynamic environment (via dynamic data and extrinsic performance parameters), occupant characteristics and occupant personalization preferences determined based on, for example, dynamically updating real-time data (see Figure 4B). Similarly, such information can also be used in other types of planning operations to adapt planned vehicle activity to real-time situations, personalize based on individual preferences, and allow the vehicle to behave more like a human driver. Below, more details are provided with reference to Figures 14-19 regarding personalized adaptive motion planning.

人类驾驶者以舒服的方式控制其车辆运动。在大多数情况下，人类驾驶者还注意与他们一起坐在车里并对车辆运动做出反应的乘员的反馈或反应。例如，某些人类驾驶者可能偏爱平稳地起动和停止车辆。在观察到坐在车内的乘员以特定方式反应时，某些经常非常突然地起动和停止车辆的人类驾驶者可能调整他们的驾驶。这样的人类行为可能在自动车辆中起到重要作用。通常，已经认识到，驾驶行为因人而异，在同一车辆中存在他人时如何调整这样的行为也可能因人而异。The human driver controls the movement of his vehicle in a comfortable way. In most cases, human drivers also pay attention to feedback or reactions from occupants who sit with them in the car and react to vehicle motion. For example, some human drivers may prefer to start and stop the vehicle smoothly. Certain human drivers, who often start and stop vehicles very abruptly, may adjust their driving upon observing the occupants of the vehicle reacting in a particular way. Such human behavior could play an important role in autonomous vehicles. In general, it has been recognized that driving behavior varies from person to person, and how to adjust such behavior in the presence of others in the same vehicle may also vary from person to person.

按照传统，自动车辆可采用这样的规划模型：其被训练为捕获普通人群的人类驾驶行为的特性。这样的一般性模型并不基于个体驾驶者/乘员偏好或意图对规划方法进行定制。本示教的目的在于，基于对于驾驶者/乘员的认识以及对于驾驶者/乘员对车辆运动的响应的动态观察，提供个性化的运动规划。Traditionally, autonomous vehicles can employ planning models that are trained to capture the characteristics of the human driving behavior of the general population. Such a generic model does not tailor the planning method based on individual driver/occupant preferences or intentions. The purpose of this teaching is to provide personalized motion planning based on driver/occupant knowledge and dynamic observation of driver/occupant responses to vehicle motion.

图14A示出了根据本示教一实施例的运动规划模块560的示例性高层次系统图。在此所示实施例中，运动规划模块560的目的在于个性化、似人型、适应性的运动规划，即，车辆的运动根据例如一般性的以及个人的喜好(其可包括已知什么是乘员偏好以及乘员对车辆当前运动有什么反应或反馈)来规划。根据本示教的运动规划模块560可包括通用运动规划器1450和乘员运动适应器1460。运动规划模块560可基于多种考虑来规划车辆运动，包括车辆所处实时情况(例如在弯曲道路上，下雨天，光线暗淡等)、车辆状况(经由内在性能参数)以及车内乘员的个人偏好(已知偏好，或是基于观察到的驾驶者反馈而动态确定的)。在给定这些考虑的情况下，车辆运动可以基于运动规划模型(其可以以适合于不同场景的方式调用)来规划。运动规划模型可包括适合用于当前给定情况的不同模型。14A shows an exemplary high-level system diagram of amotion planning module 560 in accordance with an embodiment of the present teachings. In the embodiment shown here, themotion planning module 560 is aimed at personalized, anthropomorphic, adaptive motion planning, ie, the motion of the vehicle is occupant preferences and how occupants react or feedback to the vehicle’s current motion).Motion planning module 560 in accordance with the present teachings may includegeneral motion planner 1450 andoccupant motion adaptor 1460 . Themotion planning module 560 may plan vehicle motion based on a variety of considerations, including the real-time conditions the vehicle is in (eg, on curvy roads, rainy days, dim light, etc.), vehicle conditions (via intrinsic performance parameters), and personal preferences of occupants in the vehicle (Preferences are known, or dynamically determined based on observed driver feedback). Given these considerations, vehicle motion can be planned based on a motion planning model, which can be invoked in a way that is suitable for different scenarios. The motion planning models may include different models suitable for the current given situation.

图14B示出了根据本示教一实施例的运动规划模型的示例性类型。在所示的实施例中，运动规划模型可包括通用运动规划模型(图14A中的1450)、子类别模型(图14A中的1480)或个性化模型(图14A中的1430)。通用运动规划器1450可以是基于偏好的模型，或基于影响的模型(见图14B)。基于偏好的模型可以设置为，基于关于车辆运行的一般知识，具体指定不同场景下偏爱的车辆运动。例如，当道路湿滑或结冰时，偏爱规划较慢而没有急转弯的运动。基于影响的模型可以具体指定哪种类型的运动可导致哪种类型的影响，且这样的指定可用于引导运动规划，以实现或避免特定的影响。Figure 14B illustrates an exemplary type of motion planning model in accordance with an embodiment of the present teachings. In the embodiment shown, the motion planning model may include a general motion planning model (1450 in Figure 14A), a sub-category model (1480 in Figure 14A), or a personalized model (1430 in Figure 14A). Thegeneral motion planner 1450 may be a preference-based model, or an influence-based model (see Figure 14B). A preference-based model can be set up to specify, based on general knowledge about vehicle operation, the preferred vehicle motion for different scenarios. For example, when the road is slippery or icy, there is a preference for planning slower movements without sharp turns. Influence-based models can specify which type of motion can lead to which type of influence, and such specification can be used to guide motion planning to achieve or avoid a particular influence.

相比于通用模型，用于运动规划的子类别模型可以针对车辆子类别或驾驶者/乘员的子类别。例如，子类别模型可以针对跑车，另一子类别模型可以为厢式货车提供。另外，子类别模型可以针对青少年驾驶者，另一子类别模型可针对长者。对各个子类别模型进行调节和具体指定，使得对于匹配的子类别的运动规划可以更加准确地执行。根据本示教，运动规划模型也可包括这样的个性化模型：其可包括个体模型，每一个个体模型可指定每个个体在车辆运动方面的偏好。例如，乘员个体偏好模型可指定该乘员偏爱平稳的车辆运动，另一个乘员个体偏好模型可指定某些不同的偏好。用于运动规划的这种通用、子类别以及个体模型可基于所记录的人类驾驶数据得出，故基于这样的模型规划的运动更像人类。In contrast to generic models, sub-category models for motion planning can target vehicle sub-categories or driver/occupant sub-categories. For example, a subcategory model could be for sports cars and another subcategory model could be provided for vans. Additionally, a sub-category model may be for teenage drivers and another sub-category model may be for seniors. The individual sub-category models are adjusted and specified so that motion planning for the matched sub-category can be performed more accurately. In accordance with the present teachings, the motion planning model may also include personalized models that may include individual models, each of which may specify each individual's preferences regarding vehicle motion. For example, an individual occupant preference model may specify that the occupant prefers smooth vehicle motion, and another individual occupant preference model may specify some different preference. Such generic, sub-category, and individual models for motion planning can be derived based on recorded human driving data, so motions planned based on such models are more human-like.

回到图14A，在运行中，为了实现个性化的、似人型且适应性的运动规划，运动规划模块560接收多种类型的信息，并使用不同的运动规划模型。接收的信息包括来自路径规划模块的规划路径、周围环境信息420、实时数据480、自身觉知性能参数510。基于实时数据480和自身觉知性能参数510，通用运动规划器1450确定车辆所处情况(例如下雨、黑暗等)，并在1440处相应地调用合适的通用运动规划模型，以得出通用运动规划信息。在某些实施例中，通用运动规划器1450也可确定有关的车辆和/或乘员子类别，使得相关联的子类别运动规划模块可从1480中取得并用于运动规划。通用运动规划模块1440可具体指定通用运动规划策略，例如，如果是下雪天或者车辆在弯曲道路上，优选为使得车辆运动较慢且稳定。各个子类别模型可被提供为具体指定用于子类别(例如，一种类型的车辆，例如跑车，或者，乘员的子组，例如长者)的通用运动规划策略。Returning to Figure 14A, in operation, in order to achieve personalized, human-like, and adaptive motion planning, themotion planning module 560 receives various types of information and uses different motion planning models. The received information includes the planned route from the route planning module, surroundingenvironment information 420 , real-time data 480 , and self-awareness performance parameters 510 . Based on real-time data 480 and self-awareness performance parameters 510,generic motion planner 1450 determines the situation the vehicle is in (eg, rain, darkness, etc.) and invokes appropriate generic motion planning models accordingly at 1440 to derive generic motions planning information. In some embodiments,generic motion planner 1450 may also determine relevant vehicle and/or occupant subcategories such that associated subcategory motion planning modules may be retrieved from 1480 and used for motion planning. The generalmotion planning module 1440 may specify a general motion planning strategy, for example, if it is snowing or the vehicle is on a curved road, it is preferable to make the vehicle motion slow and stable. Each subcategory model may be provided as a generic motion planning strategy specifically designated for a subcategory (eg, a type of vehicle, such as a sports car, or a subgroup of occupants, such as seniors).

由通用运动规划器1450规划的运动(基于通用运动规划模型和/或子类别运动规划模型)可以根据个性化的偏好被进一步调节或适应。在所示的实施例中，这一点由乘员运动适应器1460实现。还可能有不同的方式来将规划的运动适应为满足个性化的偏好。在某些实施例中，个性化的偏好可由个体乘员模型1430访问.如果乘员的身份是已知的，用于该乘员的相关个体乘员模型可从1430取得，且车辆运动中的具体偏好可被用于确定如何实现个性化运动规划。例如，用于特定乘员的个体模型可指示乘员偏爱平稳乘坐，而不冒风险。Motions planned by the generic motion planner 1450 (based on the generic motion planning model and/or the sub-category motion planning model) may be further adjusted or adapted according to individualized preferences. In the embodiment shown, this is accomplished by theoccupant motion adaptor 1460 . There may also be different ways to adapt the planned movement to meet individual preferences. In some embodiments, personalized preferences may be accessed byindividual occupant models 1430. If the occupant's identity is known, the associated individual occupant model for that occupant may be retrieved from 1430, and vehicle motion specific preferences may be accessed by Used to determine how to implement personalized motion planning. For example, an individual model for a particular occupant may indicate that the occupant prefers a smooth ride without taking risks.

另一种实现个性化运动规划的方式是基于动态观察的信息适应性地调节运动规划。如先前参照图4B所讨论的，实时数据480包括与乘员特性有关的信息，其可以是乘员状况、……、和/或乘员反应线索。乘员状况可能指乘员的精神、身体和功能状态。将在个性化运动规划中使用的信息也可包括关于该情况收集的其他类型的数据。乘员观察分析器1420可收集多种类型的信息，提取相关指示，于是，将这种指示发送到乘员运动适应器1460，使得这样的动态和个性化信息能在运动规划中被考虑在内。关于乘员观察分析器1420的细节参照图15-16来提供。Another way to achieve personalized motion planning is to adaptively adjust motion planning based on dynamically observed information. As previously discussed with reference to FIG. 4B , real-time data 480 includes information related to occupant characteristics, which may be occupant conditions, . . . , and/or occupant response cues. Occupant condition may refer to the mental, physical and functional state of the occupant. The information to be used in personalized motion planning may also include other types of data collected about the situation.Occupant observation analyzer 1420 may collect various types of information, extract relevant indications, and then send such indications tooccupant motion adaptor 1460 so that such dynamic and personalized information can be taken into account in motion planning. Details regarding theoccupant observation analyzer 1420 are provided with reference to Figures 15-16.

图14C示出了根据本示教一实施例在运动规划中为纳入考虑所收集的观察的示例性类型。观察可包括来自例如乘员的明确的表达，例如语音或文字输入(其可明确指示乘员想要什么)。例如，乘员可呼喊“快点！”或“我就要迟到了！”或是“我真得准时到达。”可以在运动规划中依赖检测到的这些明确表达。观察也可包括检测到的场景，其可在运动规划方面指示某些东西。场景信息可包括所涉及的事件(其可指示乘员面临的紧急事件)、乘员的当前状态(例如年龄、已知健康状况等)、一天当中的时间(其可暗示需要特定安全等级的、乘员在那时的特定任务(例如从学校接小孩))。观察也可包括可被认为是与运动规划有关的、观察到的乘员的身体反应。例如，车辆内的传感器可捕获可指示乘员情绪、乘员身体语音或乘员语调的任何数据，所有这些可反映乘员响应于当前车辆运动的想望。例如，乘员可能看起来不舒服或甚至表现出焦虑，这可能指示车辆运动对乘员来说太粗暴。尖锐的乘员语调可能指示同样的事情。特定的身体行为也可能暗示乘员对车辆运动的特定反应。例如，如果乘员正在打盹、打哈欠、看起来昏昏欲睡或正在阅读，可能指示乘员对车辆运动感到舒适。另一方面，如果观察到乘员一直在看表，这可能指示乘员感到车移动得太慢。Figure 14C illustrates an exemplary type of observations collected for consideration in motion planning according to an embodiment of the present teachings. Observations may include explicit expressions from, for example, the occupant, such as speech or text input (which may unambiguously indicate what the occupant wants). For example, the occupant can shout "Hurry up!" or "I'm going to be late!" or "I really have to be on time." These detected explicit expressions can be relied upon in motion planning. Observations can also include detected scenes, which can indicate something in terms of motion planning. Scenario information may include the event involved (which may indicate an emergency facing the occupant), the current state of the occupant (eg, age, known health conditions, etc.), the time of day (which may suggest that a particular level of safety is required, the occupant is specific tasks at that time (such as picking up kids from school). Observations may also include observed occupant physical responses that may be considered relevant to motion planning. For example, sensors within the vehicle may capture any data that may be indicative of occupant mood, occupant body voice, or occupant tone of voice, all of which may reflect the occupant's desire to respond to current vehicle motion. For example, the occupant may appear uncomfortable or even exhibit anxiety, which may indicate that the vehicle motion is too rough for the occupant. A sharp occupant tone might indicate the same thing. Certain bodily behaviors may also suggest specific occupant responses to vehicle motion. For example, if the occupant is napping, yawning, looking drowsy, or reading, it may indicate that the occupant is comfortable with vehicle motion. On the other hand, if the occupant is observed to have been looking at the watch, this may indicate that the occupant feels that the car is moving too slowly.

根据本示教，除了个性化运动规划(例如不仅关于子类别，而且关于个体)以外，运动规划也可适应于用例如自身觉知性能参数描述的当前情况和例如天气、道路状况等实时情况。乘员运动适应器1460接收来自1410的外在性能参数，并相应地规划运动。例如，如果外在性能参数指示存在太阳眩光或起雾，可相应地规划运动(例如慢下来)。According to the present teachings, in addition to personalized motion planning (eg, not only for sub-categories, but also for individuals), motion planning can also be adapted to current conditions described by eg self-awareness performance parameters and real-time conditions such as weather, road conditions, etc.Occupant motion adaptor 1460 receives extrinsic performance parameters from 1410 and plans motion accordingly. For example, if extrinsic performance parameters indicate the presence of sun glare or fog, motion can be planned accordingly (eg, slow down).

图15示出了根据本示教一实施例的乘员观察分析器1420的示例性高层次系统图。在此所示的实施例中，提供乘员观察分析器1420，用于在车辆运动方面获得乘员的动态偏好，使得运动规划可被适应于个人喜好。乘员的动态偏好基于对经由不同传感器观察到的、对于当前车辆运动的乘员反应线索的分析来取得。示例性的乘员观察分析器1420包括传感器致动器1500、多个现场传感器1510、乘员检测器1520、乘员特征检测器1540、基于视觉的反应线索推定器1580、基于听觉的反应线索推定器1590、乘员表达检测器1560、乘员场景检测器1570和用户反应生成器1595。FIG. 15 shows an exemplary high-level system diagram ofoccupant observation analyzer 1420 in accordance with an embodiment of the present teachings. In the embodiment shown here, anoccupant observation analyzer 1420 is provided for obtaining an occupant's dynamic preferences in terms of vehicle motion so that motion planning can be adapted to personal preferences. The occupant's dynamic preferences are derived based on an analysis of occupant response cues to current vehicle motion observed via different sensors. Exemplaryoccupant observation analyzer 1420 includessensor actuator 1500,multiple presence sensors 1510,occupant detector 1520,occupant feature detector 1540, vision-basedresponse cue estimator 1580, auditory-basedresponse cue estimator 1590, Anoccupant expression detector 1560, anoccupant scene detector 1570, and auser response generator 1595.

提供乘员观察分析器1420，以确定乘员对当前车辆运动的反应或反馈，从而确定车辆运动是否需要调节。例如，如果乘员反应指示乘员对当前车辆运动不愉快，调节可在运动规划中相应地做出。乘员反应将基于不同的线索来推定，包括视觉、听觉、文字或背景场景。Anoccupant observation analyzer 1420 is provided to determine occupant response or feedback to current vehicle motion to determine whether vehicle motion needs to be adjusted. For example, if occupant responses indicate that the occupant is unhappy with current vehicle motion, adjustments may be made accordingly in motion planning. Occupant responses will be inferred based on different cues, including visual, auditory, textual or background scenes.

在某些实施例中，传感器致动器1500致动现场传感器1510，以检测乘员反应。现场传感器1510包括多个传感器，包括视觉传感器、听觉传感器、红外传感器、……、或通信传感器等，其使得对乘员的任何表达的检测成为可能。例如，现场传感器1510中包括的视觉传感器可包括多个空间分布(在车辆内)的照相机装置，其能够捕获、处理并将来自多个视点的景象的图像融合为某种形式的更加有用的个体图像/视频。例如，视觉传感器可捕获乘员的手势或面部表达，其可用于推定乘员的反应。现场传感器可以被有选择地致动。例如，在夜间，为了准确观察乘员的反应，视觉传感器不能良好工作，在这种情况下，可作为替代地致动红外传感器。In certain embodiments,sensor actuator 1500 actuatesfield sensor 1510 to detect occupant responses.Presence sensors 1510 include multiple sensors, including visual sensors, auditory sensors, infrared sensors, . . . , or communication sensors, etc., that enable detection of any expression of the occupant. For example, vision sensors included infield sensors 1510 may include multiple spatially distributed (within the vehicle) camera devices capable of capturing, processing, and fusing images of the scene from multiple viewpoints into some form of more useful individual image/video. For example, vision sensors can capture occupant gestures or facial expressions, which can be used to infer occupant responses. Field sensors can be selectively actuated. For example, at night, in order to accurately observe the occupant's reaction, the visual sensor does not work well, in which case the infrared sensor can be actuated instead.

如图14C所示，多种身体反应可被观察并用于分析乘员的反应线索。乘员检测器1520接收传感器数据，并基于乘员检测模型1530来检测乘员.检测可以基于视觉或听觉信息。因此，乘员检测模型1530可包括与乘员相关联的视觉和听觉模型二者，并可被分别调用以基于单一模型数据检测乘员，或均被调用以基于视觉和听觉特征二者来检测乘员。例如，乘员检测模型1530可包括面部识别模型，其可用于基于来自一个或多于一个视觉传感器的视频或图像数据来检测乘员。乘员检测模型1530也可包括基于扬声器的乘员检测模型，通过该模型，乘员可基于其声音被识别。As shown in Figure 14C, various bodily responses can be observed and used to analyze occupant response cues.Occupant detector 1520 receives sensor data and detects occupants based onoccupant detection model 1530. Detection may be based on visual or auditory information. Thus,occupant detection model 1530 may include both visual and auditory models associated with an occupant, and may be invoked separately to detect an occupant based on a single model data, or both to detect an occupant based on both visual and auditory features. For example,occupant detection model 1530 can include a facial recognition model that can be used to detect occupants based on video or image data from one or more vision sensors. Theoccupant detection model 1530 may also include a speaker-based occupant detection model by which occupants may be identified based on their voices.

在检测到乘员时，传感器数据可以被连续馈送到乘员特征检测器1540，以检测多种乘员行为特征，其可包括视觉上的和听觉上的。例如，特定的身体语言可被检测，其可揭示乘员正在做特定的事情，例如睡觉(打盹)、阅读、打哈欠或频繁看表。这样检测的乘员特征也可包括听觉特征。例如，乘员特征检测器1540可检测乘员正在说“慢下来。”视觉和听觉线索可被同时检测，其揭示一致的反应线索。例如，乘员可一直看表并说“快点！”Upon detection of an occupant, sensor data may be continuously fed to occupantcharacteristic detector 1540 to detect a variety of occupant behavioral characteristics, which may include visual and auditory. For example, certain body language can be detected, which can reveal that the occupant is doing certain things, such as sleeping (napping), reading, yawning, or looking at a watch frequently. The occupant characteristics thus detected may also include auditory characteristics. For example, the occupantcharacteristic detector 1540 may detect that the occupant is saying "slow down." Visual and auditory cues may be detected simultaneously, revealing consistent response cues. For example, an occupant could keep looking at the watch and say "Quick!"

乘员特征可以基于视觉和听觉特征检测模型1550来检测。这样的模块可在检测何种特征方面对乘员特征检测器1540进行引导，并对将要检测的各个特征提供对应的模型，该模型能被用于检测该特征。将要检测的特征可取决于乘员，在这个意义上，可对这些模型进行个性化。例如，如果已经知道乘员是哑的，没有理由检测与该乘员相关联的听觉特征。这些特征检测模块可以是适应性的，使得一旦它们受到训练并在车辆上配置，它们可以被配置为接收计划更新或动态更新，使得模型适应于变化的情况。Occupant features may be detected based on the visual and auditoryfeature detection model 1550 . Such a module can guide theoccupant feature detector 1540 in what features to detect and provide a corresponding model for each feature to be detected that can be used to detect that feature. The features to be detected may depend on the occupant, and in this sense the models may be personalized. For example, if the occupant is already known to be mute, there is no reason to detect the auditory signature associated with the occupant. These feature detection modules can be adaptive, such that once they are trained and deployed on the vehicle, they can be configured to receive scheduled updates or dynamic updates, allowing the model to adapt to changing conditions.

于是，检测到的乘员视觉特征被发送到基于视觉的反应线索推定器1580，该推定器于是可基于这样的视觉线索来推定乘员的反应线索。例如，如果检测到乘员正在看表，基于视觉的反应推定器1580可以是这样的反应线索：乘员对车辆速度不高兴且变得不耐烦。这样的推定线索也可基于例如1550中的个性化视觉特征模型得出，该模型可用于确定这样的行为(看表)是否表示与这一特定乘员相关联的特定反应线索(可以或可以不取决于这个人)。The detected visual features of the occupant are then sent to a vision-basedresponse cue estimator 1580, which can then infer the occupant's response cues based on such visual cues. For example, if it is detected that an occupant is looking at a watch, the vision-basedresponse estimator 1580 may be a response cue that the occupant is unhappy with the vehicle speed and becomes impatient. Such putative cues can also be derived based on, for example, a personalized visual feature model in 1550, which can be used to determine whether such behavior (looking at a watch) represents a specific response cue associated with this specific occupant (which may or may not depend on the to this person).

类似地，检测到的乘员听觉特征被发送到基于听觉的反应线索推定器1590，该推定器1590于是可基于这样的听觉特征来推定乘员反应线索。例如，如果检测到乘员正在打鼾，基于听觉的反应线索推定器1590可推定为乘员对当前车辆运动感到舒适或至少不是不高兴。这样的推定线索也可基于例如1550中的个性化听觉特征模型来得出，该模型可用于确定这样的行为是否表示这一特定乘员的特定反应线索。Similarly, detected occupant auditory features are sent to auditory-basedresponse cue estimator 1590, which can then infer occupant response cues based on such auditory features. For example, if it is detected that the occupant is snoring, the auditory-basedresponse cue estimator 1590 may infer that the occupant is comfortable or at least not unhappy with the current vehicle motion. Such putative cues may also be derived based on, for example, the personalized auditory signature model in 1550, which may be used to determine whether such behaviors represent specific response cues for this particular occupant.

为了推定乘员的反应，基于视觉和基于听觉的反应线索推定器1580和1590可用于推定乘员的情绪状态。例如，从由乘员观察到的身体语言(例如焦躁不安或看上去在作呕)，可能表示乘员感到不舒服，这可能是他/她对车辆运动的反应的线索。另外，乘员在说“快点”或“放慢”时的语调也可用于推定乘员的焦虑程度，这是乘员有多不高兴的线索。这样推定的情绪状态可用于评估乘员对当前车辆运动做出响应地表现出的反应的激烈程度，并可用于引导是否和/或如何调节运动规划。To infer the occupant's response, visual-based and auditory-basedresponse cue estimators 1580 and 1590 may be used to infer the occupant's emotional state. For example, body language observed by the occupant (eg, restless or looking gagged) may indicate that the occupant is uncomfortable, which may be a clue to his/her response to vehicle motion. Additionally, the tone of voice an occupant uses when saying "hurry up" or "slow down" can also be used to infer the occupant's anxiety level, a clue to how upset the occupant is. Such an estimated emotional state can be used to assess the intensity of the occupant's response in response to current vehicle motion, and can be used to guide whether and/or how to adjust motion planning.

除了观察到的身体特征外，其他的参数也可用于推定当前车辆运动是否可接受。例如，观察来源可以是由乘员经由车内某种通信接口(例如触摸屏显示器)直接录入的输入。乘员可经由车内显示接口输入他/她想让车辆运动更为平稳。这可由乘员表达检测器1560经由不同的通信传感器来检测，这些传感器可以是文本的或听觉的。In addition to the observed physical characteristics, other parameters can also be used to infer whether the current vehicle motion is acceptable. For example, the observation source may be input entered directly by the occupant via some kind of communication interface in the vehicle (eg, a touch screen display). The occupant can input via the in-vehicle display interface that he/she wants the vehicle to move more smoothly. This may be detected by theoccupant expression detector 1560 via various communication sensors, which may be textual or audible.

如前面讨论的那样，乘员当前所处的场景也可能影响运动应当如何规划。乘员场景检测器1570被配置为检测可能与运动规划有关的任何场景参数。例如，如果已经知道每个下午在3：30pm到4：30pm(一天中的时间)之间，车辆用于从学校接孩子(手上的任务)，这可能对运动规划加以限制。也就是说，所规划的运动可能需要基于安全性。一旦检测到，这样的限制可被配置为压制所推定的(乘员)想要更快的愿望，以便保证孩子的安全。也可观察其他的场景相关因素，例如，乘员的健康和年龄。如果(由乘员模型1535)观察到乘员是位长者且罹患失智，这样的场景参数可用于避免某些检测到的乘员愿望。例如，如果当前车辆运动已经很快且乘员继续要求甚至更快，在给定乘员年龄和已知的健康状况的情况下，运动规划模块可使用这样的信息来做出合适的运动规划决策。As discussed earlier, the scene the occupant is currently in may also affect how the movement should be planned. Theoccupant scene detector 1570 is configured to detect any scene parameters that may be relevant to motion planning. For example, if it is known that each afternoon between 3:30pm and 4:30pm (time of day), the vehicle is used to pick up the child from school (task at hand), this may impose constraints on motion planning. That said, the planned movement may need to be based on safety. Once detected, such a restriction can be configured to suppress a presumed (occupant's) desire to be faster in order to keep the child safe. Other scene-related factors, such as occupant health and age, may also be observed. Such scene parameters can be used to avoid certain detected occupant wishes if the occupant is observed (by the occupant model 1535) to be elderly and suffering from dementia. For example, if the current vehicle motion is already fast and the occupant continues to demand even faster, the motion planning module may use such information to make appropriate motion planning decisions given the occupant's age and known health conditions.

由基于视觉/听觉的反应线索推定器1580和1590、乘员表达检测器1560以及乘员场景检测器1570检测到的多种乘员反应线索于是被发送到用户反应生成器1595，在那里，检测到的不同参数被选择和集成，从而生成推定的用户反应，并发送到乘员运动适应器1460，使得由通用运动规划器1450规划的运动可根据观察到的对当前车辆运动的动态用户反应而得到适应。The various occupant response cues detected by visual/auditory basedresponse cue estimators 1580 and 1590,occupant expression detector 1560, andoccupant scene detector 1570 are then sent touser response generator 1595, where the detected differences are Parameters are selected and integrated to generate putative user responses and sent tooccupant motion adaptor 1460 so that motions planned bygeneric motion planner 1450 can be adapted based on observed dynamic user responses to current vehicle motion.

图16为根据本示教一实施例用于乘员观察分析器1420的示例性过程的流程图。为了收集与乘员相关联的观察，在1610处致动合适的传感器。来自致动的传感器的信息在1620处被处理。为了确定乘员的身体行为，在1630处，基于乘员检测模型1530，对乘员进行检测。一旦乘员的身份得到确认，可获得与乘员相关联的不同类型的特征。在1640处，检测来自乘员的任何明确的表达。与乘员相关联的场景参数在1660处被检测。这样从乘员收集的明确表达以及与乘员有关的场景参数于是被发送到用户反应生成器1595。16 is a flowchart of an exemplary process foroccupant observation analyzer 1420 in accordance with an embodiment of the present teachings. To collect observations associated with the occupant, appropriate sensors are actuated at 1610 . Information from the actuated sensors is processed at 1620 . To determine the occupant's physical behavior, at 1630, based on theoccupant detection model 1530, the occupant is detected. Once the occupant's identity is confirmed, different types of characteristics associated with the occupant may be obtained. At 1640, any explicit expression from the occupant is detected. Scene parameters associated with the occupant are detected at 1660 . The articulation thus collected from the occupant and the scene parameters related to the occupant are then sent to theuser response generator 1595 .

乘员的视觉/听觉特征在1650处被检测，并被用于分别在1670和1680处推定视觉和听觉反应线索，于是，这些线索也被发送到乘员反应生成器1595。这样收集的不同类型的信息(从1640、1660、1670和1680)于是全由乘员反应生成器1595用于在1690处生成推定的用户反应。The visual/auditory characteristics of the occupant are detected at 1650 and used to infer visual and auditory response cues at 1670 and 1680, respectively, which are then also sent to theoccupant response generator 1595. The different types of information so collected (from 1640 , 1660 , 1670 and 1680 ) are then all used byoccupant response generator 1595 at 1690 to generate a putative user response.

回到图14，由乘员观察分析器1420输出的推定用户反应被发送到乘员运动适应器1460，故对于当前车辆运动的实时乘员反应可在判断如何基于来自乘员的动态反馈对规划的运动进行适应时被考虑在内。Returning to Figure 14, the putative user responses output byoccupant observation analyzer 1420 are sent tooccupant motion adaptor 1460, so real-time occupant responses to current vehicle motion can be used to determine how to adapt to planned motions based on dynamic feedback from the occupants is taken into account.

图17为根据本示教一实施例用于运动规划模块560的示例性过程的流程图。在1710和1720处，实时数据480和自身觉知性能参数分别被接收。自身觉知性能参数在1730处被处理，并被分为内在和外在性能参数。基于实时数据和内在性能参数，通用运动规划器1450在1740处生成用于车辆的规划运动。这样的规划运动基于通用运动规划模型以及任何可应用的子类别运动规划模块1480来生成。17 is a flowchart of an exemplary process formotion planning module 560 in accordance with an embodiment of the present teachings. At 1710 and 1720, real-time data 480 and self-awareness performance parameters are received, respectively. Self-awareness performance parameters are processed at 1730 and divided into intrinsic and extrinsic performance parameters. Based on real-time data and intrinsic performance parameters, thegeneral motion planner 1450 generates a planned motion for the vehicle at 1740 . Such planned motions are generated based on the generic motion planning model and any applicable sub-categorymotion planning modules 1480.

于是，为了将运动规划个性化，可基于个性化信息对通用规划运动进行适应，个性化信息可包括已知的个人偏好和动态观察的对当前车辆运动的乘员反应二者。为了实现这一点，基于个体乘员模型1430，在1750处识别已知乘员偏好。另外，在1760处，基于从不同来源/传感器收集的信息，推定动态乘员反应/反馈。或是已经知道的、或是动态推定的个人偏好于是用于对规划的运动进行个性化，例如，通过在1770处对基于通用信息规划的运动进行适应。于是，在1780处，这样的个性化规划运动作为规划运动530被输出。Thus, in order to personalize motion planning, the generic planned motion may be adapted based on personalized information, which may include both known personal preferences and dynamically observed occupant responses to current vehicle motion. To accomplish this, based on theindividual occupant model 1430, known occupant preferences are identified at 1750. Additionally, at 1760, dynamic occupant responses/feedback are inferred based on information gathered from various sources/sensors. Personal preferences, either known or dynamically inferred, are then used to personalize the planned movement, eg, by adapting at 1770 the planned movement based on the general information. Thus, at 1780, such a personalized planning motion is output asplanning motion 530.

如参照图14A所讨论的，多种模型用于模型规划，有些是通用的，有些是半通用的(子类别模型是半通用的)，有些是个性化的。除了个性化和适应性以外，这里公开的运动规划方案的目的也在于以更为似人的方式做出运动。适应于乘员动态反应可能是它的一部分。在某些实施例中，运动规划模块560使用的模型也可被生成为捕捉似人行为，故而当它们在运动规划中应用时，所规划的运动530将会更加似人化。As discussed with reference to Figure 14A, a variety of models are used for model planning, some are generic, some are semi-generic (subcategory models are semi-generic), and some are personalized. In addition to personalization and adaptation, the motion planning scheme disclosed here is also aimed at making motion in a more human-like manner. Adapting to occupant dynamics may be part of it. In some embodiments, the models used by themotion planning module 560 may also be generated to capture human-like behaviors, so when they are applied in motion planning, the plannedmotions 530 will be more human-like.

图18示出了根据本示教一实施例，用于生成这些模型的运动规划模型训练机制1800的示例性高层次系统图。在此所示实施例中，运动规划模型训练机制(MPMTM)1800包含数据预处理部分和模型训练部分。数据预处理部分包含子类别训练数据分类器1820、个体训练数据提取器1830和观察分割器1850。模型训练部分包含模型训练引擎1810和独立影响模型训练引擎1840以及独立影响模型训练引擎1840。FIG. 18 shows an exemplary high-level system diagram of a motion planningmodel training mechanism 1800 for generating these models, according to an embodiment of the present teachings. In the embodiment shown here, the motion planning model training mechanism (MPMTM) 1800 includes a data preprocessing portion and a model training portion. The data preprocessing section includes a sub-categorytraining data classifier 1820, an individualtraining data extractor 1830, and anobservation segmenter 1850. The model training part includes amodel training engine 1810 and an independent influencemodel training engine 1840 and an independent influencemodel training engine 1840.

记录的人类驾驶数据430用于对模型进行训练，故模型可捕捉更为似人化的与运动规划有关的特性。为了对通用运动规划模型1440进行训练，接收的所记录人类驾驶数据被发送到模型训练引擎1810，所训练的模型被存储为通用模型规划模型1440。为了获得子类别运动规划模型1480，所记录的人类驾驶数据430被子类别训练数据分割器1820分类为用于子类别的训练数据组，并被馈送到模型训练引擎1810，以便进行训练。对于各个子类别模型，合适的子类别训练数据组被应用，以得出对应的子类别模型，于是，这样训练的子类别模型在1480处被存储。类似地，为了获得用于运动规划的个体乘员模型1430，所记录的人类驾驶数据可被处理，以便由个体训练数据提取器1830生成不同的训练组，每个用于一个个体，并由模型训练引擎1810用于得出描述对应个体的偏好的个体乘员模型。The recordedhuman driving data 430 is used to train the model so that the model can capture more human-like characteristics related to motion planning. To train the genericmotion planning model 1440 , the received recorded human driving data is sent to themodel training engine 1810 and the trained model is stored as the genericmodel planning model 1440 . To obtain the sub-categorymotion planning model 1480, the recordedhuman driving data 430 is classified by the sub-categorytraining data segmenter 1820 into training data sets for the sub-category and fed to themodel training engine 1810 for training. For each sub-category model, the appropriate sub-category training data set is applied to derive the corresponding sub-category model, and the sub-category model thus trained is stored at 1480 . Similarly, to obtain anindividual occupant model 1430 for motion planning, the recorded human driving data can be processed to generate different training sets, each for an individual, by the individualtraining data extractor 1830, and trained by the model Theengine 1810 is used to derive an individual occupant model describing the preferences of the corresponding individual.

除了个体偏好以外，个体乘员模型1430也可包括描述由乘员反应或反馈观察到的、车辆运动对个体乘员的影响的模型。观察的反应/反馈可以是正面的或是负面的，并可用于影响将来应当如何为乘员规划运动。图19示出了根据本示教一实施例，观察到的不同类型的反应以及它们在模型训练中的作用。例如，可用于训练基于影响的模型的乘员反应/反馈可包括负面或正面的影响。乘员对特定规划运动的负面反应(负强化)可以在模型中捕获，故可在未来对这一特定乘员避免类似的运动。类似地，观察到的对规划运动的正面反应或正强化也可以在模型中捕获，用于未来的运动规划。有些反应可以是中性的，其也可由个体乘员模型捕获。In addition to individual preferences,individual occupant models 1430 may also include models that describe the effects of vehicle motion on individual occupants as observed by occupant responses or feedback. The observed reaction/feedback can be positive or negative and can be used to influence how future movements should be planned for the occupant. Figure 19 illustrates the different types of responses observed and their role in model training according to an embodiment of the present teachings. For example, occupant responses/feedback that may be used to train an impact-based model may include negative or positive impacts. An occupant's negative response to a particular planned movement (negative reinforcement) can be captured in the model so that similar movements can be avoided for this particular occupant in the future. Similarly, observed positive responses or positive reinforcement to planned motion can also be captured in the model for future motion planning. Some responses can be neutral, which can also be captured by individual occupant models.

为了获得用于个体的基于影响的模型，实时数据480——其捕捉在乘员行为、视觉、听觉线索及其状况(包括车辆运动期间的精神、身体和功能状态)方面的乘员特性——可基于个体来分割，且这样分割的数据于是可用于得出描述特定运动如何影响乘员的模型。在某些实施例中，机制1800包括观察分类器1850，其根据个体乘员对实时数据480进行分割，并将这样分割的训练数据组馈送到独立影响模型训练引擎1840，以得出个体影响模型。这样得出的个体影响模型于是存储为个体乘员模型1430的一部分。To obtain an impact-based model for an individual, real-time data 480 , which captures occupant characteristics in terms of occupant behavior, visual, and auditory cues and their conditions, including mental, physical, and functional states during vehicle motion, can be based on Individuals are segmented, and the data thus segmented can then be used to derive a model describing how a particular movement affects the occupant. In some embodiments,mechanism 1800 includesobservation classifier 1850 that segments real-time data 480 according to individual occupants, and feeds the thus segmented training data set to independent influencemodel training engine 1840 to derive individual influence models. The individual impact model thus derived is then stored as part of theindividual occupant model 1430 .

回到图5，规划模块440还包括车道规划模块570，其可用于如图20A所示的车道沿行和车道变换。车道沿行可指在车辆移动的同时保持在车道内的行为。车道变换可指在车辆移动的同时从车辆当前所处一车道移动到相邻车道的行为。车道规划可指在车道沿行或车道变换方面规划车辆行为。Returning to Figure 5, theplanning module 440 also includes alane planning module 570, which may be used for lane following and lane changes as shown in Figure 20A. Lane following may refer to the act of staying in a lane while the vehicle is moving. Lane changing may refer to the act of moving from one lane the vehicle is currently in to an adjacent lane while the vehicle is moving. Lane planning may refer to planning vehicle behavior in terms of lane following or lane changes.

图20B示出了根据本示教一实施例，与车道沿行有关的行为的示例性类型。如所示出的那样，存在多条车道(2010，2020，2030)，各条车道内的车辆可在其自己的车道内沿行。根据不同的情况，个体车辆的车道沿行行为可能不同。例如，如图所示，当车辆只是试图保持在车道内而不转弯时，车辆的行为可以是保持在车道的中央。这在图20B中关于车道2010和2020内的车辆示出。这可以称为正常行为2040。当车道2030中的车辆需要向右转弯时，例如，如图20B中的2050所示，车道2030中的车辆的行为可能不同。例如，代替保持在车道2030中央的是，该车道中的车辆可以在转弯之前驶向车道右侧，例如，使得转弯更为安全、容易。类似地，当车辆将要左转时，车道沿行行为也可以不同。车道规划模块570被配置为例如经由建模来捕捉不同情况下的车道沿行行为，使得自动驾驶车辆可以以自然且似人的方式受到控制。FIG. 20B illustrates an exemplary type of behavior related to lane following according to an embodiment of the present teachings. As shown, there are multiple lanes (2010, 2020, 2030), and vehicles in each lane can follow in their own lane. Depending on the situation, the lane following behavior of individual vehicles may vary. For example, as shown, the behavior of the vehicle may be to remain in the center of the lane when the vehicle is simply trying to stay in the lane without turning. This is shown in FIG. 20B with respect to vehicles withinlanes 2010 and 2020 . This may be referred to asnormal behavior 2040. When a vehicle inlane 2030 needs to turn right, eg, as shown at 2050 in FIG. 20B , the behavior of the vehicle inlane 2030 may be different. For example, instead of remaining centered in thelane 2030, vehicles in that lane may steer to the right of the lane before turning, eg, making the turn safer and easier. Similarly, when the vehicle is about to turn left, the lane following behavior can also be different. Thelane planning module 570 is configured to capture lane following behavior in different situations, eg, via modeling, so that the autonomous vehicle can be controlled in a natural and human-like manner.

另一方面，车道变换可涉及在车辆移动的同时从一条车道移动到相邻车道的车辆行为。不同的乘员可表现出不同的车道变换行为。出于安全考虑，对不同的情况可以有希望的车道变换行为。车道变换方面的车道规划是以安全、自然、似人、个性化的方式规划关于车道的车辆运动。Lane changes, on the other hand, may involve the behavior of a vehicle moving from one lane to an adjacent lane while the vehicle is moving. Different occupants may exhibit different lane changing behaviors. For safety reasons, there can be promising lane change behavior for different situations. Lane planning in lane changing is the planning of vehicle movement with respect to lanes in a safe, natural, human-like, and personalized way.

图20C示出了根据本示教一实施例的与车道变换有关的行为的示例性类型。示出的是不同的车道变换行为，即从当前车道2020变换到它左边的车道(车道2010)，以及从当前车道2020变换到它右边的车道(车道2030)。关于从车道2020到车道2010的车道变换，不同的车道变换行为可以在(1)做出变换有多快，以及(2)车辆以何种方式移动到下一车道的方面描述。例如，如图20B所示，示出了移动到车道2010的三种方案，它们是左车道变换行为1 2060、左车道变换行为2 2070以及左车道变换行为3 2080。分别代表移动到车辆2010的不同的速度。通过行为2060，车辆最快地移动到车道2010。通过行为2080，车辆最慢地移动到车道2010。通过行为2070移动到车道2010的速度居中。类似地，当车辆从车道2020移动到它右边的车道(2030)时，也可以有不同的车道变换行为，例如右车道变换行为1 2065、右车道变换行为2 2075以及右车道变换行为2085，如图20B所示。FIG. 20C illustrates an exemplary type of behavior related to a lane change in accordance with an embodiment of the present teachings. Shown are the different lane changing behaviors, namely changing from thecurrent lane 2020 to the lane to its left (lane 2010), and from thecurrent lane 2020 to the lane to its right (lane 2030). Regarding a lane change fromlane 2020 tolane 2010, different lane change behaviors can be described in terms of (1) how fast the change is made, and (2) how the vehicle moves to the next lane. For example, as shown in FIG. 20B , three scenarios for moving to lane 2010 are shown, which are left lane change behavior 1 2060 , left lane change behavior 2 2070 , and left lane change behavior 3 2080 . represent different speeds of movement to thevehicle 2010, respectively. Throughaction 2060, the vehicle moves tolane 2010 the fastest. Throughaction 2080 , the vehicle moves the slowest intolane 2010 . The speed of moving tolane 2010 viabehavior 2070 is centered. Similarly, when the vehicle moves fromlane 2020 to the lane to its right (2030), there can also be different lane change behaviors, such as right lane change behavior 1 2065, right lane change behavior 2 2075, and rightlane change behavior 2085, as in shown in Figure 20B.

除了车辆移动到下一车道的速度以外，车道变换行为也可在车辆如何移动到下一车道的方面不同。同样如图20B所示，当车辆将通过使用左车道变换行为1 2060从车道2020移动到车道2010时，存在由车辆使用以便缓行进入车道2010的不同的行为，例如通过沿着直线2061行驶、通过沿着曲线2062行驶(首先切入，然后打直(straight out)车辆)或通过沿着曲线2063行驶(首先向着车道2020的边缘缓行，观察，接着，在准备好时切入)。因此，关于车道变换，可以在不同的层次做出关于车辆行为的决策。In addition to the speed at which the vehicle moves to the next lane, the lane change behavior may also differ in how the vehicle moves to the next lane. As also shown in Figure 20B, when the vehicle is to move fromlane 2020 tolane 2010 by using left lane change behavior 1 2060, there are different behaviors used by the vehicle to amble intolane 2010, such as by traveling along straight line 2061, passing through Drive along curve 2062 (cut in first, then straight out vehicle) or by driving along curve 2063 (first crawl toward edge oflane 2020, watch, then, cut in when ready). Thus, with respect to lane changes, decisions about vehicle behavior can be made at different levels.

不同的驾驶者/乘员可表现出不同的车道规划(包括车道沿行和车道变换)行为，在某些情况下，同样的驾驶者/乘员可在不同的情况下有不同的行为。例如，如果街上没有人，驾驶者可决定在车道变换时迅速切入下一车道。在街道拥挤时，同样的驾驶者可能更为小心，决定花时间逐渐缓行到下一车道。车道规划模块570被配置为学习不同情况下不同的人类行为，并使用这样学习的知识/模型来获得自动驾驶中的车道规划。Different drivers/occupants may exhibit different lane planning (including lane following and lane changes) behaviors, and in some cases the same driver/occupant may behave differently in different situations. For example, if there is no one on the street, the driver can decide to quickly cut into the next lane when changing lanes. When the street is congested, the same drivers may be more careful and decide to take the time to amble gradually to the next lane. Thelane planning module 570 is configured to learn different human behaviors in different situations, and use the knowledge/models thus learned to obtain lane planning in autonomous driving.

平稳和可预测的车道沿行和车道变换行为是在自动驾驶车辆中提供似人驾驶体验的关键方面。这在车辆运行期间捕获的照相机图像和/或视频中存在明显环境噪音时可能特别困难。传统方法依赖于计算机视觉，以通过即时检测可行驶区域来检测车道。某些将端到端图像原始像素用于车辆控制信号预测。这样的传统方法不能使用所收集的可用手动驾驶数据，故它们常常产生生硬的规划和控制，容易受到环境变化的影响，同时，最终又对令人满意地运行车辆的能力产生限制。Smooth and predictable lane-following and lane-changing behavior are key aspects of delivering a human-like driving experience in autonomous vehicles. This can be particularly difficult when there is significant ambient noise in the camera images and/or video captured during vehicle operation. Traditional approaches rely on computer vision to detect lanes by instantly detecting drivable areas. Some use end-to-end image raw pixels for vehicle control signal prediction. Such traditional methods cannot use the available manual driving data collected, so they often result in blunt planning and control, are susceptible to environmental changes, and ultimately limit the ability to operate the vehicle satisfactorily.

本示教将车道检测模型和车道规划模型用于车道规划和控制。两个模型都基于大量的训练数据来训练，有些数据是加标签的，有些数据是如所收集的。对于车道检测，车道检测模型使用具有加标签车道的训练数据获得，以得出用于车道检测的监督模型(supervised model)。这样的监督模型将使用一组数量很多的训练数据来训练，这些数据覆盖大范围的环境状况，以确保训练模型的代表性和鲁棒性。This teach uses the lane detection model and the lane planning model for lane planning and control. Both models are trained based on a large amount of training data, some data is labeled and some data is collected as-is. For lane detection, a lane detection model is obtained using training data with labeled lanes to derive a supervised model for lane detection. Such a supervised model would be trained using a large set of training data covering a wide range of environmental conditions to ensure the representativeness and robustness of the trained model.

对于车道规划，为了实现似人的车道规划行为，收集大量的人类驾驶数据，并将之用于训练车道控制模型，这样的模型在用于车道规划时表现出操纵车辆时的似人行为。尽管车道检测模型和车道规划模型分别被训练，在运行中，两组模型以级联方式用于推理，以便在多种类型的环境或状况下以似人的运行模式产生具有鲁棒性的行为。在某些实施例中，当人类驾驶数据根据个体驾驶者被分类时，本示教可被配置为进一步个性化，以便产生个性化的似人车道规划模型。采用这样的个性化似人车道规划模型，取决于车辆中的乘员是谁，自动驾驶车辆可以以适应性方式在车道规划/控制中运行。For lane planning, in order to achieve human-like lane planning behavior, a large amount of human driving data is collected and used to train a lane control model, which exhibits human-like behavior when manipulating vehicles when used for lane planning. Although the lane detection model and the lane planning model are trained separately, in operation, the two sets of models are used for inference in a cascaded manner to produce robust behavior in human-like operating modes in many types of environments or situations . In certain embodiments, when human driving data is classified according to individual drivers, the present teachings may be configured to be further personalized in order to generate a personalized human-like lane planning model. With such a personalized human-like lane planning model, autonomous vehicles can operate in lane planning/control in an adaptive manner, depending on who the occupants of the vehicle are.

图21示出了根据本示教一实施例的车道规划模块570的示例性高层次系统图。在此所示实施例中，车道规划模块570包含两个模型训练引擎2110和2140，分别用于训练车道检测模型2120和车道规划模型2150。这样训练的模型于是由驾驶车道检测器2130和驾驶车道规划单元2160以级联的方式用在车道规划之中。如上面所讨论的，车道检测模型2120为监督模型，并使用具有加标签车道的训练数据进行训练。这样的监督训练数据由驾驶车道检测模型训练引擎2110处理和使用，以获得驾驶车道检测模型2120。FIG. 21 shows an exemplary high-level system diagram of alane planning module 570 in accordance with an embodiment of the present teachings. In the embodiment shown here, thelane planning module 570 includes twomodel training engines 2110 and 2140 for training thelane detection model 2120 and thelane planning model 2150, respectively. The model thus trained is then used in lane planning by thedriving lane detector 2130 and the drivinglane planning unit 2160 in a cascaded manner. As discussed above, thelane detection model 2120 is a supervised model and is trained using training data with labeled lanes. Such supervised training data is processed and used by the driving lane detectionmodel training engine 2110 to obtain the drivinglane detection model 2120.

在某些实施例中，车道检测模型2120可对应于通用模型，捕捉不同情况下的车道检测的特性。在某些实施例中，车道检测模型2120可包括不同的模型，其中的每一个可以用于在特定的不同的情况下提供检测车道的模型。例如，某些模型可以用于在正常道路状况下检测车道，某些可以用于在道路湿的时候检测车道，有些可以用于在道路有眩光或反光时检测车道，有些甚至可以用于在道路被例如雪或其他类型的视觉阻碍物体覆盖时推定车道。车道检测模型也可提供用于不同类型车辆的分立的模型。例如，有些车辆具有较高的重心，故捕捉车前地面图像的照相机可以安装在相对于地面较高的位置。在这种情况下，用于这些车辆的车道检测模型可以不同于用于照相机安装在距地平面较近水平的车辆的车道检测模型。各种类型的模型可以使用与对应场景有关的合适的加标签训练数据来训练。In some embodiments, thelane detection model 2120 may correspond to a general model, capturing the characteristics of lane detection in different situations. In some embodiments, thelane detection model 2120 may include different models, each of which may be used to provide a model for detecting lanes under certain different circumstances. For example, some models can be used to detect lanes in normal road conditions, some can be used to detect lanes when the road is wet, some can be used to detect lanes when the road has glare or reflections, and some can even be used to detect lanes when the road is wet Lanes are inferred when covered by eg snow or other types of visually obstructing objects. The lane detection model may also provide separate models for different types of vehicles. For example, some vehicles have a high center of gravity, so cameras that capture images of the ground in front of the vehicle can be mounted high relative to the ground. In this case, the lane detection model for these vehicles may be different from the lane detection model for vehicles with cameras mounted closer to the ground level. Various types of models can be trained using appropriately labeled training data relevant to the corresponding scene.

为了在自动驾驶中实现似人车道规划行为，驾驶车道规划模型训练引擎2140将记录的人类驾驶数据430用作输入，并在车道规划方面学习似人行为。如上面所讨论的，这样的人类驾驶数据可以从大范围的驾驶者/情况/状况收集，以便由驾驶车道规划模型训练引擎2140学习和捕获车道规划/控制中大范围的人类驾驶行为的特性。在某些实施例中，驾驶车道规划模型训练引擎2140可以视情况可选地将具有加标签车道的某些监督训练数据用作输入，例如，作为种子或某种小的数据组，以便使学习更快地收敛。To achieve human-like lane planning behavior in autonomous driving, the driving lane planningmodel training engine 2140 uses recordedhuman driving data 430 as input and learns human-like behavior in terms of lane planning. As discussed above, such human driving data can be collected from a wide range of drivers/situations/situations in order to be learned and captured by the driving lane planningmodel training engine 2140 to characterize a wide range of human driving behavior in lane planning/control. In some embodiments, the driving lane planningmodel training engine 2140 may optionally use as input some supervised training data with labeled lanes, eg, as a seed or some small data set, to enable learning Converge faster.

基于所记录的人类驾驶数据430，驾驶车道规划模型训练引擎2140可学习和/或训练用于车道沿行和车道变换的模型。在某些实施例中，对于车道沿行和车道变换中的每一个，可得出2150中的用于通用人类行为的通用模型。在某些实施例中，车道规划模型训练引擎2140也可学习和/或训练用于车道规划的多种模型，其中的每一个可以用于不同的已知的情况，例如，用于一般人群的特定子组的车道沿行或车道变换，或用于特定的不同驾驶环境场景(湿的道路、光线暗淡、拥挤道路)。用于一般人群的子组的这些模型也可存储在2150中。Based on the recordedhuman driving data 430, the driving lane planningmodel training engine 2140 may learn and/or train models for lane following and lane changes. In some embodiments, for each of lane following and lane change, a generic model for generic human behavior in 2150 may be derived. In certain embodiments, the lane planningmodel training engine 2140 may also learn and/or train a variety of models for lane planning, each of which may be used for a different known situation, eg, for a general population A specific subgroup of lanes along the road or lane changes, or for specific different driving environment scenarios (wet roads, dimly lit, congested roads). These models for subgroups of the general population may also be stored in 2150.

似人车道控制模型2150也可个性化并存储在2150中，当多个模型将要经由训练得出时，满足与各个不同模型相关联的状况的车道人类驾驶数据可被提取并用于对模型进行训练。例如，用于当在拥挤道路上行驶时表现出的车道相关行为的车道规划(包括车道沿行和车道变换)模型可基于与拥挤道路上的车道驾驶行为有关的人类驾驶数据来学习。用于车道规划的模型也可以是个性化的。为了实现个性化，驾驶车道规划模型训练引擎2140可基于乘员过去的驾驶数据得出用于各个个体乘员的模型(例如关于车道沿行和车道变换中的每一种)。视情况可选地，来自与乘员相关联的个人资料的信息也可在学习中使用，以便获得更为准确地反映乘员偏好的模型。The human-likelane control model 2150 can also be personalized and stored in 2150, and when multiple models are to be derived through training, lane human driving data that satisfies the conditions associated with each of the different models can be extracted and used to train the models . For example, a lane planning (including lane following and lane change) model for lane-related behavior exhibited when driving on congested roads may be learned based on human driving data related to lane driving behavior on congested roads. The models used for lane planning can also be personalized. To enable personalization, the driving lane planningmodel training engine 2140 may derive a model for each individual occupant based on the occupant's past driving data (eg, for each of lane following and lane changes). Optionally, information from profiles associated with occupants may also be used in learning to obtain models that more accurately reflect occupant preferences.

这样获得的不同类型的车道规划/控制模型于是可存储在驾驶车道控制模型存储器2150中。在某些实施例中，用于不同情况的不同的模型可被组织和编目，用于在车辆运行期间实时进行容易的识别和快速的访问。在某些实施例中，驾驶车道检测模型训练引擎2110和驾驶车道规划模型训练引擎2140可位于车辆的远程，学习可以以集中的方式进行，也就是说，它们可以基于来自不同源的训练数据来运行，学习和更新可以有规律地致动。所训练模型可被发送到分布式的车辆。在某些实施例中，基于本地获取的数据，用于车道规划的个性化模型可以在各个车辆中本地更新。The different types of lane planning/control models thus obtained may then be stored in the driving lanecontrol model memory 2150 . In some embodiments, different models for different situations may be organized and catalogued for easy identification and quick access in real-time during vehicle operation. In some embodiments, the driving lane detectionmodel training engine 2110 and the driving lane planningmodel training engine 2140 may be located remotely from the vehicle, and learning may be performed in a centralized manner, that is, they may be based on training data from different sources Running, learning and updating can be actuated regularly. The trained model can be sent to distributed vehicles. In some embodiments, the personalized model for lane planning may be updated locally in each vehicle based on locally acquired data.

经由2110和2140引擎的训练可以通过任何学习机制实现，包括人工神经网络、深度学习网络等。取决于将要获得的模型的类型和数量，各个训练引擎可包括多种子训练引擎，每个用于出于某种特定目的的特定的(一组)模型，且每个可以不同地配置和实现，以便得出最为有效的模型。除了学习以外，各个训练引擎(2110和2140)也可包括预处理机制(未示出)，用于在由学习机制使用以得到训练模型之前处理训练数据。例如，它可以包括数据分割机制，其将接收的训练数据分割为分立的组，每一个可以用于训练用于特定情况的具体模型，例如，驾驶车道规划模型训练引擎2140可被配置为得出用于一般人群的通用模型、用于车辆的驾驶者/乘员的个性化模型、用于日间光照条件下的车道规划的模型，用于夜间光照条件下的车道规划的模型、用于湿滑道路条件下的车道规划的模型、用于雪天条件的车道规划的模型。在这种情况下，预处理机制于是可首先将接收到的记录的人类驾驶数据430分组为不同的组，其中的每一个用于规划的一个模型，使得训练引擎于是可使用适合的训练数据组来学习合适的模型。在新的训练数据到来时，模型可以持续更新。模型的更新可以通过基于所接收全部数据的重新学习(成批模式(batch mode))来实现，或通过增量模式(incremental mode)来实现。Training via the 2110 and 2140 engines can be achieved by any learning mechanism, including artificial neural networks, deep learning networks, etc. Depending on the type and number of models to be obtained, each training engine may include a variety of sub-training engines, each for a particular (set of) model(s) for a particular purpose, and each may be configured and implemented differently, in order to obtain the most efficient model. In addition to learning, each training engine (2110 and 2140) may also include a preprocessing mechanism (not shown) for processing training data before being used by the learning mechanism to derive a trained model. For example, it may include a data splitting mechanism that splits the received training data into discrete groups, each of which may be used to train a specific model for a particular situation, eg, the driving lane planningmodel training engine 2140 may be configured to derive Generic model for general population, individual model for driver/occupant of vehicle, model for lane planning in daytime light conditions, model for lane planning in nighttime light conditions, for slippery A model for lane planning in road conditions, a model for lane planning in snow conditions. In this case, the preprocessing mechanism may then first group the received recordedhuman driving data 430 into different groups, one for each model of the planning, so that the training engine may then use the appropriate training data group to learn the appropriate model. The model can be continuously updated as new training data arrives. The updating of the model can be achieved by relearning based on all the data received (batch mode), or by incremental mode.

一旦生成模型(包括车道检测模型2120和驾驶车道控制模型2150)，它们用于以似人的方式并在某些情况下个性化地规划用于自动驾驶车辆的车道相关行为。如先前讨论的，在运行中，所获得的驾驶车道检测模型2120和驾驶车道控制模型2150以级联的方式应用。在所示的实施例中，当车辆在道路上时，安装在车辆中的传感器拍摄车辆当前所行驶道路的照片/视频，并将这样的传感器数据发送到驾驶车道检测器2130。除了传感器数据以外，行驶车道检测器2130也可接收自身觉知性能参数510。经由自身觉知性能参数，行驶车道检测器2130可确定多种类型的信息，例如，道路状况、车辆性能等，以便判断可以如何以合适的方式进行。例如，如果是在一天中的夜间(这可以在外在性能参数中指示)，行驶车道检测器2130可进行到调用为在黑暗光线情况下检测车道而训练的车道检测模型，以实现可靠的性能。Once the models (including thelane detection model 2120 and the driving lane control model 2150) are generated, they are used to plan lane-related behavior for autonomous vehicles in a human-like manner and in some cases individually. As previously discussed, in operation, the resulting drivinglane detection model 2120 and drivinglane control model 2150 are applied in a cascaded fashion. In the embodiment shown, when the vehicle is on the road, sensors installed in the vehicle take photos/videos of the road the vehicle is currently traveling on and send such sensor data to thedriving lane detector 2130 . In addition to sensor data, thedriving lane detector 2130 may also receive self-awareness performance parameters 510 . Via self-aware performance parameters, thedriving lane detector 2130 can determine various types of information, eg, road conditions, vehicle performance, etc., in order to determine how to proceed in an appropriate manner. For example, if it is night of the day (which may be indicated in the extrinsic performance parameter), thedriving lane detector 2130 may proceed to invoke a lane detection model trained to detect lanes in dark light conditions to achieve reliable performance.

使用适当地调用的车道检测模型，由传感器数据以及视情况可选的推定车辆位置，行驶车道检测器2130推定车道的分段。这样推定的车道分段和车辆位置于是被发送到行驶车道规划单元2160，在那里，合适的行驶车道规划模型于是以级联的方式应用，用于规划车辆的车道控制行为。Using an appropriately invoked lane detection model, thedriving lane detector 2130 estimates the segmentation of the lane from the sensor data and, optionally, the estimated vehicle position. The lane segments and vehicle positions thus estimated are then sent to the drivinglane planning unit 2160, where appropriate driving lane planning models are then applied in a cascaded manner for planning the vehicle's lane control behavior.

如先前所讨论的，车道规划包括车道沿行和车道变换二者。在运行中，车道规划涉及或是控制车辆沿行中的车辆行为、或是控制车道变换中的车辆行为。当车辆正在运动时，运行背景可以提供关于需要车道沿行还是车道变换规划的某种指示。例如，如果车辆需要驶出，其可能首先需要从不通向出口的当前车道进入驶出车道。在这种情况下，暗示着车道变换，故车道规划中涉及的任务是用于车道变换。在某些实施例中，车中乘员也可提供明确的车道控制决策，以指示车道变换，例如，通过开启转向信号。在某些实施例中，车道变换的指示也可来自车辆自身，例如，引擎可能有些问题，故自动驾驶系统可向行驶车道规划单元2160发送车道控制决策信号，指示准备车道变换，故车辆可移动到紧急车道。在正常情况下，在没有任何进入车道变换模式的指示的情况下，车辆可假设车道沿行的默认模式。As previously discussed, lane planning includes both lane following and lane changes. In operation, lane planning involves either controlling the behavior of the vehicle along the road or controlling the behavior of the vehicle during lane changes. When the vehicle is moving, the operating context can provide some indication as to whether lane following or lane change planning is required. For example, if the vehicle needs to exit, it may first need to enter the exit lane from the current lane that does not lead to the exit. In this case, lane changing is implied, so the tasks involved in lane planning are for lane changing. In some embodiments, an occupant in the vehicle may also provide an explicit lane control decision to indicate a lane change, eg, by turning on a turn signal. In some embodiments, the indication of lane change can also come from the vehicle itself, for example, there may be some problems with the engine, so the autopilot system can send a lane control decision signal to the drivinglane planning unit 2160, indicating that the lane change is ready, so the vehicle can move to the emergency lane. Under normal circumstances, without any indication to enter the lane change mode, the vehicle may assume the default mode of lane following.

为了执行车道规划，行驶车道规划单元2160从不同的来源接收多种类型的信息(例如检测到的车道，推定车辆位置，车道规划决策，自身觉知性能参数510)，并相应地进行到车道规划。例如，如果车道控制决策信号指示当前任务是用于车道沿行，用于车道沿行的模型将被取得，并用于规划。如果当前任务是车道变换，于是，用于车道变换的模型将被使用。To perform lane planning, the drivinglane planning unit 2160 receives various types of information (eg, detected lanes, estimated vehicle position, lane planning decisions, self-awareness performance parameters 510 ) from different sources, and proceeds to lane planning accordingly . For example, if the lane control decision signal indicates that the current task is for lane following, a model for lane following will be taken and used for planning. If the current task is a lane change, then the model for lane change will be used.

类似于驾驶车道检测器2130，驾驶车道规划单元2160可从2150调用通用车道规划模型，用于进行规划。其还可调用适合用于当前情况的不同的车道规划模型，以便增强性能。如早些时候讨论的那样，自身觉知性能参数510提供内在和外在性能参数二者，其可指示天气状况、道路状况等，可由驾驶车道规划单元2160用于调用合适的车道规划模型，用于进行规划。例如，如果当前任务是用于具有即将发生的右转弯的车道沿行，用于在从当前车道进行右转弯事件下的该乘员的人性化似人模型可以从2150取得，并用于规划关于如何缓行进入当前右车道中的位置并接着做出右转弯的车辆行为。Similar to thedriving lane detector 2130, the drivinglane planning unit 2160 can invoke the generic lane planning model from 2150 for planning. It can also invoke different lane planning models suitable for the current situation in order to enhance performance. As discussed earlier, the self-awareness performance parameters 510 provide both intrinsic and extrinsic performance parameters, which may be indicative of weather conditions, road conditions, etc., that may be used by the drivinglane planning unit 2160 to invoke an appropriate lane planning model, Used for planning. For example, if the current task is for a lane following with an impending right turn, a humanized anthropomorphic model for that occupant in the event of a right turn from the current lane can be taken from 2150 and used to plan on how to crawl The vehicle behavior of entering a position in the current right lane and then making a right turn.

另一方面，如果当前任务是车道变换、车道控制决策指示将要变换到当前车道左边的车道、且自身觉知性能参数指示大雨和道路淹水，那么，驾驶车道规划单元2160可适当地访问为规划在非常湿的道路上的车道变换行为所训练的车道规划模型。在某些实施例中，这些任务也可使用通用车道变换模型来实现。基于所选择的用于当前任务的模型，行驶车道规划单元2160生成规划的车道控制，于是，其可被发送到车辆控制模块450(图4A)，使得规划的车道控制行为能被实现。On the other hand, if the current task is a lane change, the lane control decision indicates to change to the lane to the left of the current lane, and the self-awareness performance parameter indicates heavy rain and road flooding, then the drivinglane planning unit 2160 may appropriately access the planning Lane planning model trained on lane changing behavior on very wet roads. In some embodiments, these tasks may also be accomplished using a generic lane change model. Based on the model selected for the current mission, the drivinglane planning unit 2160 generates planned lane controls, which can then be sent to the vehicle control module 450 (FIG. 4A) so that the planned lane control behaviors can be implemented.

行驶车道规划单元2160也可执行个性化车道规划。在某些实施例中，当前车内乘员可能是已知的，例如经由发送到行驶车道规划单元2160的驾驶者/乘员信息或经由来自传感器数据的乘员(现在未示出)的检测。在接收到这样的关于乘员的信息时，行驶车道规划单元2160可适当地调用适合于该乘员的车道控制模型。这样调用的定制化模型可以是用于乘员所属于的子组的模型，或者，可以是为该乘员个性化的模型。这样的定制化模型于是可用于控制如何以个性化方式执行车道规划。The drivinglane planning unit 2160 may also perform personalized lane planning. In some embodiments, the current vehicle occupant may be known, eg, via driver/occupant information sent to the travellane planning unit 2160 or via detection of the occupant (not shown now) from sensor data. Upon receiving such information about an occupant, the drivinglane planning unit 2160 may appropriately invoke a lane control model suitable for the occupant. The customized model so invoked may be a model for the subgroup to which the occupant belongs, or it may be a model personalized for the occupant. Such a customized model can then be used to control how lane planning is performed in a personalized manner.

图22为根据本示教一实施例，用于车道规划模块570的示例性过程的流程图。为了获得车道检测模型，具有加标签车道的训练数据首先在2210处被接收，这样接收的监督数据于是用于在2230处经由训练获得行驶车道检测模型。另一方面，为了获得行驶车道规划模型，记录的人类驾驶数据以及视情况可选的个性化资料信息在2220处被接收，并用于在2240处经由训练获得车道规划模型。一旦获得了模型，它们可基于新到来的训练数据(现在示出了)动态更新。22 is a flowchart of an exemplary process forlane planning module 570 in accordance with an embodiment of the present teachings. To obtain a lane detection model, training data with labeled lanes is first received at 2210, and the supervision data so received is then used at 2230 to obtain a driving lane detection model via training. On the other hand, to obtain a driving lane planning model, recorded human driving data and optional personalized profile information are received at 2220 and used at 2240 to obtain a lane planning model via training. Once the models are obtained, they can be dynamically updated based on newly incoming training data (now shown).

在运行过程中，当车辆正在运动时，车辆上的传感器获取包括存在车道的车前道路图像的传感器数据。这样的传感器数据在2250处接收，并在2260处用于基于车道检测模型来检测车前车道。视情况可选地，车辆的相对位置也可被推定。这样检测的车道以及视情况可选的推定车辆位置于是可被发送到行驶车道规划单元2160。在行驶车道规划单元2160中，多种类型的信息在2270处被接收，其包括车道控制决策、检测到的车道和自身觉知性能参数。这样的信息用于确定将要使用的车道规划模型，使得车道规划可在2280处基于合适地选择的车道规划模型实现。During operation, when the vehicle is moving, sensors on the vehicle acquire sensor data including images of the road ahead of the vehicle where lanes exist. Such sensor data is received at 2250 and used at 2260 to detect a lane ahead of the vehicle based on a lane detection model. Optionally, the relative position of the vehicle may also be estimated. The lanes thus detected, and optionally the estimated vehicle position, can then be sent to the travellane planning unit 2160 . In the drivinglane planning unit 2160, various types of information are received at 2270, including lane control decisions, detected lanes, and self-awareness performance parameters. Such information is used to determine the lane planning model to be used so that lane planning can be implemented at 2280 based on an appropriately selected lane planning model.

通过由人类驾驶数据进行学习，习得的车道规划模型捕获车道规划中的人类行为的特性，故而当这样的模型在自动驾驶中使用时，车辆可以以似人的方式受到控制。另外，通过进一步基于乘员/驾驶者的相关驾驶数据对车道规划模型进行个性化，车辆的车道规划行为可以以车内乘员/驾驶者熟悉且舒服的方式受到控制。By learning from human driving data, the learned lane planning model captures the characteristics of human behavior in lane planning, so when such a model is used in autonomous driving, the vehicle can be controlled in a human-like manner. Additionally, by further personalizing the lane planning model based on the relevant driving data of the occupant/driver, the lane planning behavior of the vehicle can be controlled in a way that is familiar and comfortable to the occupant/driver in the vehicle.

参照图5-22，公开了规划模块440关于路径规划、运动规划和车道规划的细节。规划模块440的输出包括来自路径规划模块550的规划路径520、来自运动规划模块560的规划运动530、来自车道规划模块570的规划车道控制540(见图5)。这样的输出可以被发送到自动驾驶车辆的不同部分，以便执行规划的车辆行为。例如，规划路径520可被发送到负责在路径控制方面引导车辆的车辆部分(例如内置GPS)。规划运动530和规划车道控制540可被发送到车辆控制模块450(在图4中)，故关于运动和车道控制的规划车辆行为可以经由车辆控制模块450在车辆上执行。5-22, details of theplanning module 440 regarding path planning, motion planning, and lane planning are disclosed. The output ofplanning module 440 includes plannedpath 520 frompath planning module 550,planned motion 530 frommotion planning module 560, plannedlane control 540 from lane planning module 570 (see Figure 5). Such outputs can be sent to different parts of the self-driving vehicle in order to execute the planned vehicle behavior. For example, theplanned route 520 may be sent to a vehicle portion (eg, built-in GPS) responsible for guiding the vehicle in route control.Planned motion 530 and plannedlane control 540 may be sent to vehicle control module 450 (in FIG. 4 ), so planned vehicle behavior with respect to motion and lane control may be performed on the vehicle viavehicle control module 450 .

当运动和车道控制被规划为实现似人行为时，车辆控制模块450的目标在于传送规划的动作。根据本示教，车辆控制模块450的目标还在于学习如何根据在车辆怎样做出动作或响应不同情况下的不同控制信号的方面的知识来控制车辆，故车辆可被控制为实现希望的效果，包括规划的车辆行为。传统的方法应用基于机器学习的控制，并由经典的机制得出车辆动态模型，这些经典机制常常不能对真实世界中发生的多种情况建模。结果，常常导致不好的性能，且在某些情况下，可能导致危险的结果。尽管某些传统方法被设计为从历史数据经由例如精神网络来学习车辆动态模型，能够在普通场景下学习车辆动态模型，在某些情况下，这样的系统做出了有着实质性、不可预测的错误的预测，这在真实生活中可能是致命的。When motion and lane control are planned to achieve human-like behavior, the goal of thevehicle control module 450 is to deliver the planned action. According to the present teachings, thevehicle control module 450 also aims to learn how to control the vehicle based on knowledge of how the vehicle behaves or responds to different control signals in different situations, so the vehicle can be controlled to achieve the desired effect, Include planned vehicle behavior. Traditional approaches apply machine learning-based control and derive vehicle dynamics models from classical mechanisms that often fail to model the many situations that occur in the real world. The result is often poor performance and, in some cases, potentially dangerous results. Although some traditional methods are designed to learn vehicle dynamics models from historical data via, for example, mental networks, which can learn vehicle dynamics models in common scenarios, in some cases such systems make substantial and unpredictable behaviors. False predictions, which can be fatal in real life.

本示教公开了一种方法，其使得实现准确的仿真和车辆运行安全成为可能。取代直接从历史数据学习车辆动态模型的是，经典机制模型用作支柱模型，并学习如何由历史数据调节预测结果。另外，对将被做出的调节的限制被明确规定为防止显著偏离正常情况的预测结果的方法。The present teaching discloses a method that enables accurate simulation and safety of vehicle operation. Instead of learning a vehicle dynamics model directly from historical data, a classical mechanism model is used as the backbone model and learns how to adjust the prediction results by the historical data. In addition, restrictions on the adjustments to be made are specified as a means of preventing significant deviations from the predicted outcome of the normal situation.

图23A示出了用于生成车辆控制信号的传统方法的系统图。为了确定控制车辆实现特定目标运动需要的车辆控制信号，提供车辆运动学模型2310，且其由车辆运动学模型(VKM)车辆控制信号生成器2320基于目标运动和关于当前车辆状态的信息来使用。例如，如果当前车辆状态是30英里每小时，且目标运动在接下来的5秒内达到40英里每小时，VKM车辆控制信号生成器2320使用这样的信息来基于车辆运动学模型2310确定何种车辆控制将被应用，使得加速度能使车辆实现目标运动。图23A所示方法基于传统车辆运动学模型2310，其仅仅是机械动态模型。FIG. 23A shows a system diagram of a conventional method for generating vehicle control signals. To determine the vehicle control signals needed to control the vehicle to achieve a particular target motion, avehicle kinematic model 2310 is provided and used by a vehicle kinematic model (VKM) vehiclecontrol signal generator 2320 based on the target motion and information about the current vehicle state. For example, if the current vehicle state is 30 mph and the target motion reaches 40 mph within the next 5 seconds, the VKM vehiclecontrol signal generator 2320 uses such information to determine what kind of vehicle based on thevehicle kinematics model 2310 Control will be applied so that the acceleration enables the vehicle to achieve the target movement. The method shown in Figure 23A is based on a conventionalvehicle kinematics model 2310, which is only a mechanical dynamic model.

图23B示出了根据本示教一实施例，图4A中的车辆控制模型450的高层次系统图。这里公开的车辆控制模块450的目标在于提供生成车辆控制信号的能力，这样的信号可以使得用于自动车辆的似人驾驶行为成为可能。如所示的，车辆控制模块450包括似人车辆控制单元2340和似人车辆控制(HLVC)模型2330。为了实现似人自动驾驶，似人车辆控制单元2340接收记录的人类驾驶数据430，以便将之用于学习似人车辆控制，并生成描述似人车辆控制行为的HLVC模型2330。FIG. 23B shows a high-level system diagram of thevehicle control model 450 of FIG. 4A in accordance with an embodiment of the present teachings. The goal of thevehicle control module 450 disclosed herein is to provide the ability to generate vehicle control signals that may enable human-like driving behavior for autonomous vehicles. As shown, thevehicle control module 450 includes a human-likevehicle control unit 2340 and a human-like vehicle control (HLVC)model 2330 . In order to achieve human-like autonomous driving, the human-likevehicle control unit 2340 receives recordedhuman driving data 430 to use it to learn human-like vehicle control and generate anHLVC model 2330 that describes the human-like vehicle control behavior.

在创建了HLVC模型2330的情况下，当似人车辆控制单元2340接收关于目标运动和当前车辆状态的信息时，其基于HLVC模型2330，关于与车辆相关联的实时情况(由实时数据480描绘)，生成似人车辆控制信号。当另外的所记录的人类驾驶数据430可用时，HLVC模型2330可被动态更新或重新训练，使它捕捉多种情况下人类车辆控制行为的特性。HLVC模型2330的动态更新可经由图23B所示的模型更新信号来触发。当满足特定条件时(例如，具有预定间隔的有规律更新设置，或当用于更新的附加数据达到某个等级时)，模型更新信号可被手动或自动触发。With theHLVC model 2330 created, when the humanoidvehicle control unit 2340 receives information about the target motion and current vehicle state, it is based on theHLVC model 2330 about the real-time conditions associated with the vehicle (depicted by real-time data 480 ) , which generates a human-like vehicle control signal. As additional recordedhuman driving data 430 becomes available, theHLVC model 2330 can be dynamically updated or retrained so that it captures the characteristics of human vehicle control behavior in a variety of situations. The dynamic update of theHLVC model 2330 can be triggered via the model update signal shown in Figure 23B. The model update signal may be triggered manually or automatically when certain conditions are met (eg, regular update settings with predetermined intervals, or when additional data for the update reaches a certain level).

在某些实施例中，HLVC模型2330也可被个性化。这一点在图23C中示出。这种情况下的HLVC模型2330可包括多种HLVC子模型(例如2330-1，2330-2，……，2330-N)，其中的每一个可对应于例如具有类似特征的子人群，并基于与该子人群有关的所记录的人类驾驶数据430的一部分来训练。例如，HLVC子模型可涉及偏爱谨慎驾驶的人的子人群，故模型可以基于这样的训练数据得出：其为在来自表现出谨慎驾驶记录的对应子人群的驾驶记录的所记录人类驾驶数据430中的训练数据。如果应用合适的训练数据，以得出个性化的子模型，HLVC子模型也可被个性化(例如子模型用于个体)。In some embodiments, theHLVC model 2330 may also be personalized. This is shown in Figure 23C. TheHLVC model 2330 in this case may include multiple HLVC sub-models (eg, 2330-1, 2330-2, ..., 2330-N), each of which may correspond, for example, to subpopulations with similar characteristics and based on trained on a portion of the recordedhuman driving data 430 related to the subpopulation. For example, the HLVC sub-model may relate to a sub-population of people who prefer to drive cautiously, so the model may be derived based on training data that is recordedhuman driving data 430 on the driving records from the corresponding sub-population exhibiting a record of cautious driving training data in . HLVC sub-models can also be personalized (eg, sub-models for individuals) if appropriate training data is applied to derive personalized sub-models.

下面参照图24-29来公开关于似人车辆控制单元2340的细节。图24示出了似人车辆控制单元2340的示例性内部高层次架构，其包括似人车辆控制模型生成器2410和似人车辆控制信号生成器2420。似人车辆控制模型生成器2410将记录的人类驾驶数据430用作输入，并将该信息用于学习和训练HLVC模型2330。从用于训练的所记录人类驾驶数据430提取的示例性类型的数据可包括例如应用于车辆的车辆控制数据和车辆状态，车辆状态可包括车辆控制数据被应用之前和之后的车辆状态。Details regarding the humanoidvehicle control unit 2340 are disclosed below with reference to FIGS. 24-29. FIG. 24 shows an exemplary internal high-level architecture of the humanoidvehicle control unit 2340 , which includes a humanoid vehiclecontrol model generator 2410 and a humanoid vehiclecontrol signal generator 2420 . The humanoid vehiclecontrol model generator 2410 uses the recordedhuman driving data 430 as input and uses this information to learn and train theHLVC model 2330. Exemplary types of data extracted from recordedhuman driving data 430 for training may include, for example, vehicle control data applied to the vehicle and vehicle state, which may include vehicle state before and after the vehicle control data is applied.

将用于得出HLVC模型2330的数据也可包括环境数据，其描述车辆控制数据达到对应的车辆状态的周围状况。环境数据可包括多种类型的信息，例如，道路状况、天气状况、车辆类型和状况。在某些实施例中，环境数据也可包括关于车内乘员的信息以及乘员的特性，例如性别、年龄、健康情况、偏好等。来自人类驾驶数据的所有这些不同类型的信息可呈现某些变化，这些变化可能影响乘员的车辆控制行为。例如，当道路湿或者滑的时候，人类驾驶者可能在制动车辆方面表现出与道路不滑时不同的车辆控制行为(例如更慢地在制动器上施压)。The data that will be used to derive theHLVC model 2330 may also include environmental data describing the surrounding conditions for which the vehicle control data achieves the corresponding vehicle state. Environmental data may include various types of information, such as road conditions, weather conditions, vehicle types and conditions. In some embodiments, the environmental data may also include information about the occupants in the vehicle and the characteristics of the occupants, such as gender, age, health, preferences, and the like. All of these different types of information from human driving data can present certain changes that may affect the vehicle control behavior of the occupants. For example, when the road is wet or slippery, a human driver may exhibit different vehicle control behavior in braking the vehicle than when the road is not slippery (eg, applying pressure on the brakes more slowly).

当HLVC模型2330被生成时，其可由似人车辆控制信号生成器2420用于在接收到希望的目标运动时生成车辆控制信号，以便在实现希望的目标运动时得到似人车辆控制行为。为了产生似人车辆控制信号，车辆控制信号生成器2420获得实时数据480，并在调用HLVC模型2330中使用该数据来生成似人车辆控制信号，实时数据480包括关于希望的目标运动时车辆周围的信息。如上面的实例所示，目标运动可以是在5秒内将车辆从30英里每小时的当前速度加速到40英里每小时。该时刻的实时数据可能指示车辆位于的道路具有陡坡且道路由于当前正在下雨而变滑。这样的实时数据是有关的，并可作为环境数据被提供给HLVC模型2330。似人车辆控制信号生成器2420可调用具有这样的参数的HLVC模型2330，以便获得推理出的似人车辆控制信号，该信号使得自动车辆以类似于人类驾驶的方式实现希望的目标运动。When theHLVC model 2330 is generated, it can be used by the humanoid vehiclecontrol signal generator 2420 to generate vehicle control signals upon receipt of the desired target motion in order to obtain a humanoid vehicle control behavior when the desired target motion is achieved. To generate the human-like vehicle control signal, the vehiclecontrol signal generator 2420 obtains real-time data 480, which includes information about the surroundings of the vehicle as the desired target moves, and uses this data in invoking theHLVC model 2330 to generate the human-like vehicle control signal. information. As shown in the example above, the target motion may be to accelerate the vehicle from its current speed of 30 mph to 40 mph in 5 seconds. Real-time data at this moment may indicate that the road the vehicle is on has a steep slope and that the road is slippery because it is currently raining. Such real-time data is relevant and can be provided to theHLVC model 2330 as environmental data. The human-like vehiclecontrol signal generator 2420 may invoke theHLVC model 2330 with such parameters to obtain inferred human-like vehicle control signals that cause the autonomous vehicle to achieve the desired target motion in a manner similar to human driving.

图25为根据本示教一实施例的似人车辆控制单元2340的示例性过程的流程图。为了生成HLVC模型2330，所记录的人类驾驶数据在2510中被接收，以获得训练数据，该数据在2520处用在训练过程中，以便在2530处得出HLVC模型2330。这样生成的HLVC模型2330于是在请求的目标运动在2550处被接收时用在运行之中。基于请求的目标运动，似人车辆控制信号生成器2420在2560处获得实时数据480，以便提取在该时刻与车辆有关的信息，包括环境数据、当前车辆状态等。在某些实施例中，也可获得(未示出)与车内乘员有关的信息(例如乘员的身份或乘员的特性)，使得这样的信息可用于对生成合适的似人车辆控制信号的过程进行个性化。FIG. 25 is a flowchart of an exemplary process of the humanoidvehicle control unit 2340 in accordance with an embodiment of the present teachings. To generate theHLVC model 2330 , recorded human driving data is received at 2510 to obtain training data, which is used in the training process at 2520 to derive theHLVC model 2330 at 2530 . TheHLVC model 2330 thus generated is then used in operation when the requested target motion is received at 2550. Based on the requested target motion, the humanoid vehiclecontrol signal generator 2420 obtains real-time data 480 at 2560 to extract information related to the vehicle at that time, including environmental data, current vehicle status, and the like. In certain embodiments, information related to the occupants of the vehicle (eg, the identity of the occupant or the characteristics of the occupant) is also available (not shown) so that such information can be used to inform the process of generating appropriate humanoid vehicle control signals Personalize.

似人车辆控制信号生成器2420获得的信息于是可应用到HLVC模型2330，以便在2570处根据HLVC模型2330生成似人车辆控制信号。在个性化时，适合该情况的、一个或多于一个的具体HLVC子模型可被调用，并用于生成个性化的似人车辆控制信号。在运行期间，似人车辆控制单元2340可在2540处检查是否存在更新触发信号。如果在2540处判断为接收到更新信号，似人车辆控制模型生成器2410进行到步骤2510，以收集训练数据，并基于动态收集的人类驾驶数据适应性地调节或是重新训练HLVC模型2330。The information obtained by the human-like vehiclecontrol signal generator 2420 may then be applied to theHLVC model 2330 to generate human-like vehicle control signals according to theHLVC model 2330 at 2570 . At the time of personalization, one or more specific HLVC sub-models suitable for the situation may be invoked and used to generate personalized human-like vehicle control signals. During operation, the humanoidvehicle control unit 2340 may check at 2540 for an update trigger signal. If it is determined at 2540 that an update signal is received, the human-like vehiclecontrol model generator 2410 proceeds to step 2510 to collect training data and adaptively adjust or retrain theHLVC model 2330 based on dynamically collected human driving data.

图26示出了根据本示教一实施例，似人车辆控制模型生成器2410的示例性高层次系统图。在所示出的实施例中，似人车辆控制模型生成器2410包括训练数据处理单元2610、VKM车辆控制预测引擎2630和车辆控制模型学习引擎2640。训练数据处理单元2610将记录的人类驾驶数据430用作输入，对该输入进行处理以生成将被用于训练HLVC模型2330的训练数据2620。记录的人类驾驶数据430可包括环境数据2620-1、车辆状态数据2620-2、……、以及当前车辆控制数据2620-3。环境数据2620-1可包括例如道路状况等的信息，诸如道路坡度、转弯角度、表面滑性条件、道路湿润度等等。环境数据也可包括与对车辆的限制有关的信息，例如速度限制、一天中的时间、季节、位置等等。这样的环境数据可用作训练中的背景信息，故HLVC模型2330可学习不同背景条件下的似人车辆控制行为，使得训练模型描绘不同情况下表现的似人车辆控制行为。如果需要个性化，获得的训练数据也可分类为不同的训练数据子组(未示出)，其中的每一组可用于对针对属于该组的乘员特有的HLVC子模型进行训练。FIG. 26 shows an exemplary high-level system diagram of a humanoid vehiclecontrol model generator 2410 in accordance with an embodiment of the present teachings. In the illustrated embodiment, the humanoid vehiclecontrol model generator 2410 includes a trainingdata processing unit 2610 , a VKM vehiclecontrol prediction engine 2630 and a vehicle controlmodel learning engine 2640 . The trainingdata processing unit 2610 uses the recordedhuman driving data 430 as input, which is processed to generatetraining data 2620 that will be used to train theHLVC model 2330. The recordedhuman driving data 430 may include environmental data 2620-1, vehicle status data 2620-2, . . . , and current vehicle control data 2620-3. The environmental data 2620-1 may include, for example, information such as road conditions, such as road grades, turning angles, surface slippery conditions, road wetness, and the like. Environmental data may also include information related to restrictions on the vehicle, such as speed limits, time of day, season, location, and the like. Such environmental data can be used as background information in training, so theHLVC model 2330 can learn the human-like vehicle control behavior under different background conditions, so that the training model depicts the human-like vehicle control behavior exhibited in different situations. If personalization is desired, the obtained training data can also be classified into different training data subgroups (not shown), each of which can be used to train a HLVC submodel specific to the occupant belonging to that group.

车辆状态数据2620-2可包括对车辆状态进行描绘的信息，包括例如车辆位置、车辆速度、车辆横滚角/俯仰角/航向角以及车辆的转向角等等。车辆控制数据2620-3可提供描绘应用到车辆的控制的信息，例如用特定力施加的制动、通过用特定角度旋转方向盘施加的转向、或节气门。Vehicle state data 2620-2 may include information delineating the state of the vehicle, including, for example, vehicle position, vehicle speed, vehicle roll/pitch/yaw angles, vehicle steering angle, and the like. Vehicle control data 2620-3 may provide information depicting controls applied to the vehicle, such as braking applied with a specific force, steering applied by rotating the steering wheel with a specific angle, or throttle.

根据本示教，代替训练HLVC模型2330以直接生成车辆控制信号的是，在如何调节使用传统运动学模型预测的车辆控制信号方面，本示教将传统的基于运动学模型的预测方法与通过由人类驾驶数据学习产生的学习模型结合或融合，使得调节得到似人的车辆控制行为。这样的一体化方法不仅使得更为准确的车辆控制成为可能，还使车辆控制的似人感成为可能。In accordance with the present teachings, instead of training theHLVC model 2330 to directly generate vehicle control signals, in how to adjust the vehicle control signals predicted using traditional kinematic models, the present teachings combine traditional kinematic model-based prediction methods with The learning models generated by human driving data learning are combined or fused, so that human-like vehicle control behavior can be adjusted. Such an integrated approach not only enables more accurate vehicle control, but also a human-like feel to vehicle control.

在学习HLVC模型2330时，车辆状态数据2620-2和车辆控制数据2620-3被提供给VKM车辆控制预测引擎2630，以便预测由于所运用的控制而实现的运动。VKM车辆控制预测引擎2630基于车辆运动学模型2310执行预测(例如，经由传统的机械动态方法)，以生成基于VKM的预测信号，如图26所示。基于VKM的预测于是被发送到车辆控制模型学习引擎2640，其将与来自训练数据2620的其他信息相结合，以便进行学习。基于VKM车辆控制预测引擎2630的输出和训练数据2620，车辆控制模型训练引擎2640在例如迭代过程中对HLVC模型2330进行训练，一直到HLVC模型2330收敛。一旦模型2339收敛，其用于得出在给定希望目标运动情况下的似人车辆控制信号。In learning theHLVC model 2330, vehicle state data 2620-2 and vehicle control data 2620-3 are provided to the VKM vehiclecontrol prediction engine 2630 in order to predict motion due to the applied controls. The VKM vehiclecontrol prediction engine 2630 performs predictions (eg, via conventional machine dynamics methods) based on thevehicle kinematics model 2310 to generate a VKM-based prediction signal, as shown in FIG. 26 . The VKM-based predictions are then sent to the vehicle controlmodel learning engine 2640, which will be combined with other information from thetraining data 2620 for learning. Based on the output of the VKM vehiclecontrol prediction engine 2630 and thetraining data 2620, the vehicle controlmodel training engine 2640 trains theHLVC model 2330, eg, in an iterative process, until theHLVC model 2330 converges. Once the model 2339 has converged, it is used to derive the humanoid vehicle control signal given the desired target motion.

如图所示，车辆控制模型学习引擎2640可被模型更新信号触发。当其被致动时，车辆控制模型学习引擎2640调用训练数据处理单元2610和VKM车辆控制预测引擎2630，以便初始化训练过程。在某些实施例中，任何后续的基于另外的人类驾驶数据的训练可以以衍生(derivative)方式进行或者以成批模式运行，即，重新训练HLVC模型2330。As shown, the vehicle controlmodel learning engine 2640 may be triggered by a model update signal. When activated, the vehicle controlmodel learning engine 2640 invokes the trainingdata processing unit 2610 and the VKM vehiclecontrol prediction engine 2630 in order to initialize the training process. In some embodiments, any subsequent training based on additional human driving data may be performed in a derivative fashion or run in a batch mode, ie, retraining theHLVC model 2330.

图27为根据本示教一实施例，用于似人车辆控制模型生成器2410的示例性过程的流程图。记录的人类驾驶数据430首先在2710处被接收。接收的人类驾驶数据在2720处被处理，以获得训练数据。某些训练数据在2730处用于基于传统的车辆运动学模型2310生成基于VKM的预测。对于基于学习和融合的训练，训练数据的多种方面在2740处被识别，并在2750处用于训练HLVC模型2330。在收敛之后，HLVC模型2330在2760处被创建。27 is a flowchart of an exemplary process for a humanoid vehiclecontrol model generator 2410 in accordance with an embodiment of the present teachings. Loggedhuman driving data 430 is first received at 2710 . The received human driving data is processed at 2720 to obtain training data. Certain training data is used at 2730 to generate VKM-based predictions based on conventionalvehicle kinematic models 2310 . For learning and fusion based training, aspects of the training data are identified at 2740 and used at 2750 to train theHLVC model 2330. After convergence, theHLVC model 2330 is created at 2760.

图28示出了根据本示教一实施例的似人车辆控制信号生成器2420的示例性高层次系统图。如这里所讨论的，似人车辆控制信号生成器2420目的在于，对于具体的目标运动，基于HLVC模型2330，生成似人车辆控制信号，使得当似人车辆控制信号用于控制车辆时，车辆表现出似人的车辆控制行为。在这一实施例中，似人车辆控制信号生成器2420包括VKM车辆控制信号推理引擎2810、背景数据确定器2820以及基于HLVC模型的融合单元2830。FIG. 28 shows an exemplary high-level system diagram of a human-like vehiclecontrol signal generator 2420 in accordance with an embodiment of the present teachings. As discussed herein, the humanoid vehiclecontrol signal generator 2420 aims to generate, for a specific target motion, based on theHLVC model 2330, a humanoid vehicle control signal such that the vehicle behaves when the humanoid vehicle control signal is used to control the vehicle. Human-like vehicle control behavior. In this embodiment, the humanoid vehiclecontrol signal generator 2420 includes a VKM vehicle controlsignal inference engine 2810, acontextual data determiner 2820, and an HLVC model-basedfusion unit 2830.

在运行中，在接收到目标运动时，似人车辆控制信号，VKM车辆控制信号推理引擎2810获得车辆的当前状态，并基于车辆运动学模型2310来生成基于VKM的车辆控制信号。如这里讨论的那样，使用传统方法来生成仅仅基于车辆运动学模型2310的推理车辆控制信号的目的在于在一开始提供纯粹基于机械动态的推理车辆控制信号。为了在获得目标运动的车辆控制中实现似人行为，基于所推理的VKM的车辆控制信号将被进一步用作到基于HLVC模型的融合单元2830的输入，在那里，基于VKM的车辆控制信号被用作原始推理结果，该结果将与基于HLVC的方法融合，使得基于VKM的车辆控制信号可根据学习的HLVC模型2330受到调节.In operation, upon receiving the target motion, the human-like vehicle control signal, the VKM vehicle controlsignal inference engine 2810 obtains the current state of the vehicle and generates a VKM-based vehicle control signal based on thevehicle kinematics model 2310. As discussed herein, the purpose of using conventional methods to generate inferred vehicle control signals based solely on thevehicle kinematics model 2310 is to initially provide inferred vehicle control signals based purely on mechanical dynamics. To achieve human-like behavior in vehicle control to obtain target motion, the inferred VKM-based vehicle control signal will be further used as input to the HLVC model-basedfusion unit 2830, where the VKM-based vehicle control signal is used with As the original inference results, the results will be fused with the HLVC-based method, so that the VKM-based vehicle control signals can be conditioned according to the learnedHLVC model 2330.

在接收到目标运动时，基于HLVC模型的融合单元2830可致动背景数据确定器2820，以获得与车辆周围有关的任何信息。背景数据确定器2820接收实时数据480并提取相关信息，例如环境数据或乘员数据等，并发送到基于HLVC模型的融合单元2830。基于目标运动、当前车辆状态、车辆周围的背景信息和使用传统车辆运动学模型2310推理的基于VKM的车辆控制信号，基于HLVC模型的融合单元2830基于这样的输入数据访问HLVC模型2330，以获得融合的似人车辆控制信号。Upon receiving object motion, the HLVC model basedfusion unit 2830 may actuate thecontextual data determiner 2820 to obtain any information related to the surroundings of the vehicle. Thecontextual data determiner 2820 receives the real-time data 480 and extracts relevant information, such as environmental data or occupant data, etc., and sends it to thefusion unit 2830 based on the HLVC model. The HLVC model basedfusion unit 2830 accesses theHLVC model 2330 based on such input data to obtain a fusion based on the target motion, current vehicle state, background information around the vehicle, and VKM-based vehicle control signals inferred using the conventionalvehicle kinematics model 2310 The humanoid vehicle control signal.

如这里所讨论的，HLVC模型2330可通过学习在基于VKM模型的预测和来自所记录人类驾驶数据430的观察信息之间的差异来创建。如此，HLVC模型2330捕获和学习到的东西可对应于将要对基于VKM的车辆控制信号做出的调节，从而实现似人行为。如先前所讨论的，由于学习过程可能产生过度拟合的情况、特别是在训练数据包括离群值时，为使车辆控制中由于对基于VKM的车辆控制信号的风险最小化，通过基于某些融合限制2840来限制对VKM车辆控制信号的调节，似人车辆控制信号生成器2420也可视情况可选地包括预防措施，如图28所示。通过这种方式，作为基于修改的VKM的车辆控制信号生成的似人车辆控制信号可以使人类行为的可能性最大化，但又使车辆控制中的风险最小化。As discussed herein, theHLVC model 2330 may be created by learning the differences between the predictions based on the VKM model and the observed information from the recordedhuman driving data 430 . As such, what theHLVC model 2330 captures and learns may correspond to adjustments to be made to the VKM-based vehicle control signals to achieve human-like behavior. As previously discussed, since the learning process may produce overfitting, especially when the training data includes outliers, to minimize the risk in vehicle control due to VKM-based vehicle control signals, the Incorporating the constraints 2840 to limit the adjustment of the VKM vehicle control signals, the humanoid vehiclecontrol signal generator 2420 may also optionally include precautions, as shown in FIG. 28 . In this way, human-like vehicle control signals generated as modified VKM-based vehicle control signals can maximize the likelihood of human behavior, yet minimize risks in vehicle control.

在某些实施例中，关于车内乘员的信息也可从实时数据480中被提取，并可用于访问与该乘员相关联的个性化HLVC子模型，其可以为用于该乘员所属于的组的HLVC子模型或对于该乘员完全个性化的HLVC子模型。使用这样的个性化HLVC子模型可允许似人车辆控制信号生成器2420生成个性化的似人车辆控制信号，使得基于它进行的车辆控制可以不仅仅是似人的，还是乘员个人喜好的。In some embodiments, information about an occupant in the vehicle may also be extracted from real-time data 480 and used to access a personalized HLVC submodel associated with the occupant, which may be for the group to which the occupant belongs HLVC submodel or a fully personalized HLVC submodel for that occupant. Using such a personalized HLVC sub-model may allow the humanoid vehiclecontrol signal generator 2420 to generate a personalized humanoid vehicle control signal so that vehicle control based on it can be not only humanlike, but also occupant personal preference.

图29为根据本示教一实施例的似人车辆控制信号生成器2420的示例性过程的流程图。目标运动信息和车辆状态数据首先在2910处被接收。基于目标运动和车辆状态，车辆运动学模型2310在2920处被访问，并在2930处用于推理VKM车辆控制信号。这种基于机械动态模型推理的控制信号被发送到基于HLVC模型的融合单元2830。为了获得融合的似人车辆控制信号，背景数据确定器2820在2940处接收实时数据数据480，并在2950处提取与车辆有关的相关信息。使用背景信息以及VKM车辆控制信号，基于HLVC模型的融合单元2830在2960处基于HLVC模型2330推理似人车辆控制信号。这样推理的似人车辆控制信号于是在2970处输出，使得实现目标运动的似人车辆控制可以以似人的方式实现。FIG. 29 is a flowchart of an exemplary process of a human-like vehiclecontrol signal generator 2420 in accordance with an embodiment of the present teachings. Target motion information and vehicle state data are first received at 2910 . Based on the target motion and vehicle state, avehicle kinematics model 2310 is accessed at 2920 and used at 2930 to infer the VKM vehicle control signals. This control signal based on the inference of the mechanical dynamic model is sent to thefusion unit 2830 based on the HLVC model. To obtain the fused human-like vehicle control signal, thecontextual data determiner 2820 receives the real-time data data 480 at 2940 and extracts relevant information about the vehicle at 2950 . Using the context information and the VKM vehicle control signals, the HLVC model basedfusion unit 2830 infers the humanoid vehicle control signals based on theHLVC model 2330 at 2960 . The human-like vehicle control signal so reasoned is then output at 2970 so that the human-like vehicle control that achieves the target motion can be implemented in a human-like manner.

图30示出了可用于实现实施本示教的特定系统的移动装置的架构。此移动装置3000包括但不限于智能电话、平板电脑、音乐播放器、手持式游戏机、全球定位系统(GPS)接收器和可穿戴计算装置(例如眼镜、腕表等)，或者出于任何其他的形态。此实例中的移动装置3000包括：一个或多于一个的中央处理单元(CPU)3040；一个或多于一个的图形处理单元(GPU)3030；存储器3060；通信平台3010，例如无线通信模块；存储器3090；一个或多于一个的输入/输出(I/O)装置3050；用于基于视觉的呈现的显示器或投影仪3020-a；以及，一个或多于一个的多模态接口通道3020-b。多模态通道可包括听觉通道或用于发信号或通信的其它介质通道。任何其他合适的部件——包括但不限于系统总线或控制器(未示出)——也可包括在移动装置3000中。如图30所示，移动操作系统3070(例如iOS、Android、WindowsPhone等)以及一个或多于一个的应用3080可从存储器3090载入到存储器3060中，以便由CPU 3040执行。30 illustrates the architecture of a mobile device that can be used to implement a particular system implementing the present teachings. Suchmobile devices 3000 include, but are not limited to, smartphones, tablets, music players, handheld game consoles, global positioning system (GPS) receivers, and wearable computing devices (eg, glasses, wristwatches, etc.), or for any other Shape. Themobile device 3000 in this example includes: one or more than one central processing unit (CPU) 3040; one or more than one graphics processing unit (GPU) 3030;memory 3060; acommunication platform 3010, eg, a wireless communication module;memory 3090; one or more input/output (I/O)devices 3050; a display or projector for vision-based presentation 3020-a; and, one or more multimodal interface channels 3020-b . Multimodal channels may include auditory channels or other media channels for signaling or communication. Any other suitable components, including but not limited to a system bus or controller (not shown), may also be included inmobile device 3000. As shown in FIG. 30 , a mobile operating system 3070 (eg, iOS, Android, Windows Phone, etc.) and one ormore applications 3080 may be loaded frommemory 3090 intomemory 3060 for execution byCPU 3040 .

为了实现本公开中介绍的多种模块、单元及其功能，计算机硬件平台可用作用于这里介绍的一个或多于一个元件的硬件平台。硬件元件、操作系统和这种计算机的编程语言在性质上是传统的，且假设本领域技术人员足够熟悉它们，以便使这些技术适应于这里介绍的本示教。具有用户接口元件的计算机可用于实现个人计算机(PC)或其他类型的工作站或终端装置，但是，如果合适地编程的话，计算机也可作为服务器运行。据信，本领域技术人员熟悉这种计算机设备的结构、编程和一般运行，因此，附图可能是不言自明的。In order to implement the various modules, units and their functions described in this disclosure, a computer hardware platform may be used as the hardware platform for one or more of the elements described herein. The hardware elements, operating system, and programming language of such a computer are conventional in nature, and it is assumed that those skilled in the art are sufficiently familiar with them to adapt these techniques to the present teachings presented herein. A computer with user interface elements may be used to implement a personal computer (PC) or other type of workstation or terminal device, but the computer may also operate as a server if suitably programmed. It is believed that those skilled in the art are familiar with the structure, programming and general operation of such computer equipment and therefore the drawings may be self-explanatory.

图31示出了能用于实现实施本示教的特定系统的计算装置的架构。实现本示教的这种特定系统具有硬件平台的功能框图，该硬件平台包括用户接口元件。计算机可以是通用计算机或专用计算机。二者都能用于实施用于本示教的特定系统。这种计算机3100可用于实现如这里所介绍的本示教的任何部件或实施形态。尽管为方便起见示出了仅仅一个这样的计算机，与这里介绍的本示教有关的计算机功能可以以分布式方式在若干个类似的平台上实现，从而分散处理负荷。31 illustrates the architecture of a computing device that can be used to implement a particular system implementing the present teachings. This particular system implementing the present teachings has a functional block diagram of a hardware platform that includes user interface elements. The computer can be a general purpose computer or a special purpose computer. Both can be used to implement the specific system used for this teaching. Such acomputer 3100 may be used to implement any component or implementation of the present teachings as described herein. Although only one such computer is shown for convenience, computer functions related to the present teachings presented herein may be implemented in a distributed fashion across several similar platforms, thereby spreading the processing load.

例如，计算机3100包括与连接于其上的网络相连接的COM端口3150，以促进数据通信。计算机3100还包括中央处理单元(CPU)3120，其采用一个或多于一个处理器的形式，用于执行程序指令。示例性计算机平台包括：内部通信总线3110；不同形式的程序存储器和数据存储器，例如盘3170、只读存储器(ROM)3130或随机访问存储器(RAM)3140，用于将要由计算机处理和/或进行通信的多种数据文件以及将由CPU执行的可能的程序指令。计算机2600还包括I/O部件3160，其支持在计算机和这里的其他部件(例如接口元件3180)之间不同介质形式的输入/输出流。接口元件的示例性类型可对应于在自动驾驶车辆上配置的不同类型的传感器3180-a。另一类型的接口元件可对应于显示器或投影仪3180-b，用于基于视觉的通信。还可以有用于其他多模态接口通道的附加部件，例如：听觉装置3180c，其用于基于音频的通信；和/或，部件2680-d，用于基于通信发信号，例如，使得车辆部件(例如车辆座椅)振动的信号。计算机3100也可经由网络通信接收编程和数据。For example, thecomputer 3100 includes aCOM port 3150 connected to a network connected thereto to facilitate data communication.Computer 3100 also includes a central processing unit (CPU) 3120 in the form of one or more processors for executing program instructions. Exemplary computer platforms include: aninternal communication bus 3110; various forms of program memory and data memory, such asdisk 3170, read only memory (ROM) 3130, or random access memory (RAM) 3140, for processing and/or processing to be performed by the computer. Various data files for communication and possible program instructions to be executed by the CPU. Computer 2600 also includes an I/O component 3160 that supports input/output streams in various media formats between the computer and other components herein (eg, interface element 3180). Exemplary types of interface elements may correspond to the different types of sensors 3180-a deployed on an autonomous vehicle. Another type of interface element may correspond to a display or projector 3180-b for vision-based communication. There may also be additional components for other multimodal interface channels, such as: hearing device 3180c for audio-based communications; and/or component 2680-d for signaling based on communications, for example, to cause vehicle components ( such as vehicle seats) vibration signals.Computer 3100 may also receive programming and data via network communications.

因此，如上面所概述的本示教的方法的实施形态可以在程序中实现。本技术的程序方面可被看作典型地出于可执行代码和/或相关数据的形式的“产品”或“制品”，该可执行代码和/或相关数据被承载在一种机器可读介质上或在其中实现。有形非暂时性“存储器”类型介质包括任何或全部存储器或其他的用于计算机、处理器等的存储器或其相关模块，例如多种半导体存储器、带驱动器、盘驱动器等，其可在任何时候提供用于软件编程的存储。Accordingly, embodiments of the method of the present teachings as outlined above may be implemented in a program. The program aspects of the present technology may be viewed as a "product" or "article of manufacture", typically in the form of executable code and/or related data carried on a machine-readable medium on or in it. Tangible non-transitory "memory" type media includes any or all memory or other memory or associated modules thereof for computers, processors, etc., such as various semiconductor memories, tape drives, disk drives, etc., which may be provided at any time Storage for software programming.

所有或部分软件有时可通过网络(例如互联网或多种其他电信网络)传送。例如，这种传送可使软件从一台计算机或处理器向另一台(例如从管理服务器或搜索引擎操作者的主机或其他增强广告服务器向着计算环境的硬件平台或实现与本示教相关的计算环境或类似功能的其他系统)的载入成为可能。因此，可承载软件元件的另一类型的介质包括光、电和电磁波，例如通过本地装置之间的物理接口、通过有线和光固定网络、通过多种空中链路使用。承载这种波的物理元件(例如有线或无线链路，光链路等)也被看作承载软件的介质。如这里所使用的，除了限制为有形的“存储”介质，例如计算机或机器“可读介质”的术语指参与向处理器提供指令以便执行的任何介质。All or part of the software may sometimes be delivered over a network such as the Internet or various other telecommunications networks. For example, such transfer may enable software from one computer or processor to another (eg, from a management server or a search engine operator's host computer or other enhanced advertising server to a hardware platform of the computing environment or to implement a hardware platform related to the present teachings). A computing environment or other system of similar functionality) becomes possible. Thus, another type of medium that can carry software elements includes optical, electrical, and electromagnetic waves, eg, through physical interfaces between local devices, through wired and optical fixed networks, through the use of various air links. The physical elements that carry such waves (eg wired or wireless links, optical links, etc.) are also considered as the medium that carries the software. As used herein, other than being limited to tangible "storage" media, terms such as computer or machine "readable media" refer to any medium that participates in providing instructions to a processor for execution.

因此，机器可读介质可采用多种形式，包括但不限于有形存储介质、载波介质或物理传输介质。非易失性存储介质包括例如光或磁盘，例如任何计算机等等之中的任何存储装置，其可用于实现附图所示的系统或其任何部件。易失性存储介质包括动态存储器，例如这种计算机平台的主存储器。有形传输介质包括：同轴电缆、铜线和光纤，其包括构成计算机系统内的总线的导线。载波传输介质可采用电或电磁信号或者是声或光波(例如在射频(RF)和红外(IR)数据通信期间生成的那些)的形式。计算机可读介质的一般形式因此包括例如软盘、可折叠盘、硬盘、磁带、任何其他磁介质、CD-ROM、DVD或DVD-ROM、任何其他光介质、穿孔卡片纸带、具有孔的图案的任何其他物理存储介质、RAM、PROM和EPROM、闪速EPROM、任何其他的存储器芯片或插装盒、载波传输数据或指令、传送这样的载波的链路或电缆、或计算机可从之读取编程代码和/或数据的任何其他介质。许多这些形式的计算机可读介质可以涉入将一个或多于一个的指令的一个或多于一个的序列承载到物理处理器，以便执行。Thus, a machine-readable medium may take many forms, including, but not limited to, tangible storage media, carrier wave media, or physical transmission media. Non-volatile storage media includes, for example, optical or magnetic disks, any storage device such as any computer, or the like, which can be used to implement the system shown in the figures, or any of its components. Volatile storage media include dynamic memory, such as the main memory of such a computer platform. Tangible transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up a bus within a computer system. Carrier-wave transmission media may take the form of electrical or electromagnetic signals, or acoustic or light waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer readable media thus include, for example, floppy disks, foldable disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs or DVD-ROMs, any other optical media, punched cardstock tape, Any other physical storage medium, RAM, PROM and EPROM, flash EPROM, any other memory chips or cartridges, carrier waves transporting data or instructions, links or cables carrying such carrier waves, or from which a computer can read programming any other medium of code and/or data. Many of these forms of computer-readable media can be involved in carrying one or more sequences of one or more instructions to a physical processor for execution.

本领域技术人员将会明了，本示教适用于多种修改和/或增强。例如，尽管上面介绍的多种部件的实现可以在硬件装置中实现，其还可实现为仅仅使用软件的解决方案，例如安装在已有的服务器上。另外，这里所公开的本示教也实现为固件、固件/软件组合、固件/硬件组合或是硬件/固件/软件组合。It will be apparent to those skilled in the art that the present teachings are applicable to various modifications and/or enhancements. For example, although the implementation of the various components described above may be implemented in hardware devices, it may also be implemented as a software-only solution, such as installed on an existing server. In addition, the present teachings disclosed herein may also be implemented as firmware, a firmware/software combination, a firmware/hardware combination, or a hardware/firmware/software combination.

尽管上面已经介绍了本示教和/或其他实例，将会明了，可对之做出多种修改，且这里公开的主题可以以多种形式和实例实现，且本示教可以在多种应用中应用，这里仅仅介绍了其中的一些。所附权利要求旨在要求落入本示教真实范围内的任何以及全部应用、修改和变型。Although the present teachings and/or other examples have been described above, it will be apparent that various modifications may be made thereto, that the subject matter disclosed herein may be embodied in various forms and examples, and that the present teachings may be used in various applications. applications, only a few of them are described here. The appended claims are intended to claim any and all applications, modifications, and variations that fall within the true scope of the present teachings.

Claims (21)

1. A method implemented on a computer having at least one processor, memory, and a communication platform for path planning for an autonomous vehicle, comprising:

obtaining information about an origin position and a destination position, wherein the destination position is a place to which the autonomous vehicle is to drive;

identifying one or more available paths between the origin location and the destination location;

obtaining a self-awareness performance model instantiated based on the one or more available paths, wherein the self-awareness performance model predicts the operating performance of the autonomous vehicle based on the one or more available paths; and

a planned path for the autonomous vehicle between the origin location and the destination location is selected from the one or more available paths based on the self-awareness performance model.

2. The method of claim 1, wherein the self-awareness performance model includes an intrinsic performance model specifying at least one intrinsic performance parameter that limits the operational capability of the autonomous vehicle due to conditions internal to the autonomous vehicle and an extrinsic performance model specifying at least one extrinsic performance parameter that limits the operational capability of the autonomous vehicle due to conditions external to the autonomous vehicle.

3. The method of claim 2, wherein the self-aware performance model comprises any one of a parametric model, a descriptive model, a probabilistic model, and combinations thereof.

4. The method of claim 1, wherein the self-awareness performance model is dynamically updated to generate an updated self-awareness performance model, wherein the updated self-awareness performance model reflects a scene with which the autonomous vehicle is currently associated.

5. The method of claim 4, wherein the updating of the self-awareness performance model is triggered by an event comprising a scheduled time, the origin location being updated, the destination location being updated, the one or more available paths being updated, and a request to update the self-awareness performance model being received.

6. The method of claim 2, wherein the selecting step comprises:

identifying a set of candidate paths from the one or more available paths based on an intrinsic performance model;

determining preferences of occupants within the autonomous vehicle in terms of a path taken by the autonomous vehicle to a destination location; and

a planned path is identified from the set of candidate paths based on an extrinsic performance model and the preferences of the occupant.

7. The method of claim 6, wherein determining the preference comprises:

obtaining recorded human driving data, the data including driving data associated with the occupant; and

generating a preference for the occupant that is personalized based on driving data associated with the occupant.

8. A machine-readable non-transitory medium having data recorded thereon for path planning for an autonomous vehicle, wherein the data, once read by a machine, causes the machine to perform:

obtaining information about an origin position and a destination position, wherein the destination position is a place to which the autonomous vehicle is to drive;

identifying one or more available paths between the origin location and the destination location;

obtaining a self-awareness performance model instantiated based on the one or more available paths, wherein the self-awareness performance model predicts the operating performance of the autonomous vehicle based on the one or more available paths; and

a planned path for the autonomous vehicle between the origin location and the destination location is selected from the one or more available paths based on the self-awareness performance model.

9. The medium of claim 8, wherein the self-awareness performance model includes an intrinsic performance model specifying at least one intrinsic performance parameter that limits the operational capability of the autonomous vehicle due to conditions internal to the autonomous vehicle and an extrinsic performance model specifying at least one extrinsic performance parameter that limits the operational capability of the autonomous vehicle due to conditions external to the autonomous vehicle.

10. The medium of claim 9, wherein the self-aware performance model comprises any one of a parametric model, a descriptive model, a probabilistic model, and combinations thereof.

11. The medium of claim 8, wherein the self-awareness performance model is dynamically updated to generate an updated self-awareness performance model, wherein the updated self-awareness performance model reflects a scene with which the autonomous vehicle is currently associated.

12. The medium of claim 11, wherein the updating of the self-awareness performance model is triggered by an event comprising a scheduled time, the origin location being updated, the destination location being updated, the one or more available paths being updated, and a request to update the self-awareness performance model being received.

13. The medium of claim 9, wherein the selecting step comprises:

identifying a set of candidate paths from the one or more available paths based on an intrinsic performance model;

determining preferences of occupants within the autonomous vehicle in terms of a path taken by the autonomous vehicle to a destination location; and

a planned path is identified from the set of candidate paths based on an extrinsic performance model and the preferences of the occupant.

14. The medium of claim 13, wherein determining the preference comprises:

obtaining recorded human driving data, the data including driving data associated with the occupant; and

generating a preference for the occupant that is personalized based on driving data associated with the occupant.

15. A system for path planning for an autonomous vehicle, comprising:

an interface unit configured to obtain information about an origin position and a destination position, wherein the destination position is a place where an autonomous vehicle is to drive;

a global path planner configured to identify one or more available paths between an origin location and a destination location; and

a path selection engine configured to:

obtaining a self-awareness performance model instantiated based on the one or more available paths, wherein the self-awareness performance model predicts the operating performance of the autonomous vehicle based on the one or more available paths; and

a planned path for the autonomous vehicle between the origin location and the destination location is selected from the one or more available paths based on the self-awareness performance model.

16. The system of claim 15, wherein the self-awareness performance model includes an intrinsic performance model specifying at least one intrinsic performance parameter that limits the operational capability of the autonomous vehicle due to conditions internal to the autonomous vehicle and an extrinsic performance model specifying at least one extrinsic performance parameter that limits the operational capability of the autonomous vehicle due to conditions external to the autonomous vehicle.

17. The system of claim 16, wherein the self-awareness performance model comprises any one of a parametric model, a descriptive model, a probabilistic model, and combinations thereof.

18. The system of claim 15, wherein the self-awareness performance model is dynamically updated to generate an updated self-awareness performance model, wherein the updated self-awareness performance model reflects a scene with which the autonomous vehicle is currently associated.

19. The system of claim 18, wherein the updating of the self-awareness performance model is triggered by an event comprising a scheduled time, the origin location being updated, the destination location being updated, the one or more available paths being updated, and a request to update the self-awareness performance model being received.

20. The system of claim 16, wherein the global path planner comprises:

an occupant preference determiner configured to determine preferences of occupants within the autonomous vehicle in terms of a path taken by the autonomous vehicle to a destination location; and

a path selection engine configured to

Identifying a set of candidate paths from the one or more available paths based on an intrinsic performance model, an

A planned path is identified from the set of candidate paths based on an extrinsic performance model and the preferences of the occupant.

21. The system of claim 20, wherein the global path planner further comprises:

an occupant driving data analyzer configured to analyze recorded human driving data, the data including driving data associated with the occupant;

a preference personalization module configured to generate personalized routing preferences for the occupant; and

an occupant preference determiner configured to generate a preference of the occupant personalized based on driving data associated with the occupant.

Images