Movatterモバイル変換

[0]ホーム
URL:

CN109543651B - A method for detecting driver's dangerous driving behavior - Google Patents

A method for detecting driver's dangerous driving behaviorDownload PDF

Info

Publication number
CN109543651B
CN109543651BCN201811487007.5ACN201811487007ACN109543651BCN 109543651 BCN109543651 BCN 109543651BCN 201811487007 ACN201811487007 ACN 201811487007ACN 109543651 BCN109543651 BCN 109543651B
Authority
CN
China
Prior art keywords
driver
steering wheel
standard
head
neck
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201811487007.5A
Other languages
Chinese (zh)
Other versions
CN109543651A (en
Inventor
吴骅跃
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changan University
Original Assignee
Changan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changan UniversityfiledCriticalChangan University
Priority to CN201811487007.5ApriorityCriticalpatent/CN109543651B/en
Publication of CN109543651ApublicationCriticalpatent/CN109543651A/en
Application grantedgrantedCritical
Publication of CN109543651BpublicationCriticalpatent/CN109543651B/en
Expired - Fee Relatedlegal-statusCriticalCurrent
Anticipated expirationlegal-statusCritical

Links

Images

Classifications

Landscapes

Abstract

The invention discloses a method for detecting dangerous driving behaviors of a driver, which can obtain various parameters of a steering wheel, realize the adaptive detection of the parameters of the steering wheel of various vehicles and does not need any manual parameter input. After the vehicle steering wheel data is obtained through the depth data and the steering wheel is modeled, the driving behavior can be detected without any sensor and data interface, so that the detection of the driving behavior is greatly facilitated; can begin to detect driver's driving behavior fast, and the detection condition does not receive ambient light to influence, and to a great extent has promoted the feasibility that driver dangerous behavior detected based on vision.

Description

Translated fromChinese
一种驾驶员危险驾驶行为检测方法A method for detecting driver's dangerous driving behavior

技术领域technical field

本发明涉及行为检测方法，具体涉及一种驾驶员危险驾驶行为检测方法。The invention relates to a behavior detection method, in particular to a driver's dangerous driving behavior detection method.

背景技术Background technique

在国内外较前沿的研究中，主要针对驾驶员的疲劳状态和驾驶行为进行检测和分析，通过分析驾驶员的精神状态和行为来分析驾驶员的行为是最直观有效的方法，这是当前主要的研究方向。研究主要分为：基于驾驶员生理状态的驾驶行为检测、基于车辆的运行状态的驾驶行为检测和通过计算机视觉技术来检测驾驶员的行为。In the cutting-edge research at home and abroad, the detection and analysis of the driver's fatigue state and driving behavior are mainly carried out. It is the most intuitive and effective method to analyze the driver's behavior by analyzing the driver's mental state and behavior. research direction. The research is mainly divided into: the driving behavior detection based on the driver's physiological state, the driving behavior detection based on the vehicle's running state, and the detection of the driver's behavior through computer vision technology.

目前，比较常用的生理状态有：EEG(脑电信号)、EGG(心电信号)、EMG(肌肉电信号)，但是要想测量这些参数，需要通过驾驶员穿戴专门的测量仪器，一方面可能会影响驾驶员的驾驶操作，从而影响测量结果的准确性，另一方面这类设备造价比较高，增加设备成本，不适用在实际中使用；基于车辆的运行状态的驾驶行为检测这种方法主要通过在车辆上安装各种传感器或使用CAN总线和OBD接口来获取行驶过程中的车辆的参数，根据得到的参数来判断驾驶员是否出现了违规驾驶行为，这种方法需要在车辆上安装大量的传感器，影响驾驶员正常驾驶且系统过于复杂，有可能对车辆进行改造，成本比较高，而且由于驾驶员的驾驶行为存在行为差异，不容易统一建模，使得检测结果存在误差。At present, the more commonly used physiological states are: EEG (electroencephalogram signal), EGG (cardiogram signal), EMG (muscle electrical signal), but in order to measure these parameters, it is necessary for the driver to wear a special measuring instrument. On the one hand, it may It will affect the driver's driving operation, thereby affecting the accuracy of the measurement results. On the other hand, the cost of this type of equipment is relatively high, which increases the cost of the equipment and is not suitable for practical use; the method of driving behavior detection based on the running state of the vehicle is mainly By installing various sensors on the vehicle or using the CAN bus and OBD interface to obtain the parameters of the vehicle during driving, and judge whether the driver has violated the driving behavior according to the obtained parameters, this method needs to install a large number of sensors on the vehicle. Sensors affect the normal driving of the driver and the system is too complex, it is possible to modify the vehicle, the cost is relatively high, and because the driver's driving behavior is different, it is not easy to model uniformly, resulting in errors in the detection results.

发明内容Contents of the invention

本发明的目的在于提供一种驾驶员危险驾驶行为检测方法，用以解决现有技术在对驾驶员驾驶行为进行检测时影响了驾驶员的正常行驶，导致检测结果准确率不高等问题。The purpose of the present invention is to provide a driver's dangerous driving behavior detection method, which is used to solve the problems of the prior art that the detection of the driver's driving behavior affects the driver's normal driving, resulting in low accuracy of detection results.

为了实现上述任务，本发明采用以下技术方案：In order to achieve the above tasks, the present invention adopts the following technical solutions:

一种驾驶员危险驾驶行为检测方法，所述的方法包括：A method for detecting a driver's dangerous driving behavior, the method comprising:

步骤1、驾驶员双手握住方向盘的两侧保持标准姿态时，采用Kinect摄像头采集驾驶员此时的骨骼姿态，建立标准姿态模型；Step 1. When the driver holds both sides of the steering wheel with both hands to maintain a standard posture, use the Kinect camera to collect the skeleton posture of the driver at this time, and establish a standard posture model;

所述的建立标准姿态模型，包括：The described establishment of standard attitude model includes:

步骤11、建立如式I所示的方向盘标准包络空间，所述方向盘标准包括空间在X轴、Y轴以及Z轴上的坐标的单位均为单位长度：Step 11, establish the steering wheel standard envelope space shown in formula I, the steering wheel standard includes the coordinates of the space on the X-axis, Y-axis and Z-axis in units of unit length:

Figure BDA0001894766580000021
Figure BDA0001894766580000021

其中，(Xlefthand,Ylefthand,Zlefthand)为驾驶员在保持标准姿态时的左手骨三维坐标，(Xrighthand,Yrighthand,Zrighthand)为驾驶员在保持标准姿态时的右手骨三维坐标，SWxmin为方向盘标准包络空间X轴向上坐标最小值，SWxmax为方向盘标准包络空间X轴向上坐标最大值，SWymin为方向盘标准包络空间Y轴向上坐标最小值，SWymax为方向盘标准包络空间Y轴向上坐标最大值，SWzmin为方向盘标准包络空间Z轴向上坐标最小值，SWzmax为方向盘标准包络空间Z轴向上坐标最大值，Lfxp为方向盘的长度在方向盘标准包络空间中X轴向上的投影距离，Hfxp为方向盘的高在方向盘标准包络空间中Y轴向上的投影距离，Wfxp为方向盘的宽度在方向盘标准包络空间中Z轴向上的投影距离；Among them, (Xlefthand , Ylefthand , Zlefthand ) is the three-dimensional coordinates of the driver's left hand bone when maintaining a standard posture, (Xrighthand , Yrighthand , Zrighthand ) is the three-dimensional coordinates of the driver's right hand bone when maintaining a standard posture, SWxmin is the minimum value of the upper coordinate of the X-axis in the standard envelope space of the steering wheel, SWxmax is the maximum value of the upper coordinate of the X-axis in the standard envelope space of the steering wheel, SWymin is the minimum value of the upper coordinate of the Y-axis in the standard envelope space of the steering wheel, and SWymax is the maximum value of the upper coordinate of the Y axis in the standard envelope space of the steering wheel, SWzmin is the minimum value of the upper coordinate of the Z axis in the standard envelope space of the steering wheel, SWzmax is the maximum value of the upper coordinate of the Z axis in the standard envelope space of the steering wheel, and Lfxp is the maximum value of the upper coordinate of the Z axis in the standard envelope space of the steering wheel. The projection distance of the length of the length of the steering wheel on the X axis in the standard envelope space of the steering wheel, Hfxp is the projection distance of the height of the steering wheel on the Y axis in the standard envelope space of the steering wheel, Wfxp is the width of the steering wheel in the standard envelope space of the steering wheel The projection distance on the middle Z axis;

步骤12、采用式V计算方向盘标准转向频率ffxp：Step 12. Calculate the standard steering frequency ffxp of the steering wheel using formula V:

Figure BDA0001894766580000031
Figure BDA0001894766580000031

其中，Rfxp为方向盘的半径在方向盘标准包络空间中X轴向上的投影距离，

Figure BDA0001894766580000032
单位为单位长度，SRO为急转向检测灵敏度，SRO>0,Tfxp为方向盘正常转向半圈需要的时间，单位为s；Among them, Rfxp is the projection distance of the radius of the steering wheel on the X-axis in the standard envelope space of the steering wheel,
Figure BDA0001894766580000032
The unit is the unit length, SRO is the sharp steering detection sensitivity, SRO >0, Tfxp is the time required for the steering wheel to turn half a circle normally, and the unit is s;

步骤13、采用式VI计算驾驶员肩部标准向量VJB：Step 13. Calculate the driver's shoulder standard vector VJB using formula VI:

Figure BDA0001894766580000033
Figure BDA0001894766580000033

其中，(Xleftshoulder,Yleftshoulder,Zleftshoulder)为驾驶员在保持标准姿态时的左肩关节三维坐标，(Xrightshoulder,Yrightshoulder,Zrightshoulder)为驾驶员在保持标准姿态时的右肩关节三维坐标；Among them, (Xleftshoulder , Yleftshoulder , Zleftshoulder ) is the three-dimensional coordinates of the driver's left shoulder joint when maintaining a standard posture, (Xrightshoulder , Yrightshoulder , Zrightshoulder ) is the three-dimensional coordinates of the driver's right shoulder joint when maintaining a standard posture ;

步骤14、采用式VII计算驾驶员头颈部标准向量VNH：Step 14. Calculate the standard vector VNH of the driver's head and neck using formula VII:

VNH＝(Xhead-Xneck,Yhead-Yneck,Zhead-Zneck) 式VIIVNH ＝(Xhead -Xneck ,Yhead -Yneck ,Zhead -Zneck ) formula VII

其中，(Xhead,Yhead,Zhead)为驾驶员在保持标准姿态时的头部三维坐标，(Xneck,Yneck,Zneck)为驾驶员在保持标准姿态时的颈部三维坐标；Among them, (Xhead , Yhead , Zhead ) are the three-dimensional coordinates of the driver's head when maintaining a standard posture, and (Xneck , Yneck , Zneck ) are the three-dimensional coordinates of the driver's neck when maintaining a standard posture;

步骤2、在驾驶员驾驶过程中重复执行本步骤，检测驾驶员在驾驶过程中是否具有危险驾驶行为：Step 2. Repeat this step during the driving process of the driver to detect whether the driver has dangerous driving behavior during the driving process:

利用Kinect摄像头采集驾驶员的骨骼姿态，与所述的标准姿态模型进行对比，检测驾驶员是否具有危险驾驶行为，包括：Use the Kinect camera to collect the driver's skeletal posture, compare it with the standard posture model, and detect whether the driver has dangerous driving behavior, including:

步骤21、根据驾驶员左手骨的三维坐标与右手骨的三维坐标是否均在所述的包络空间内判断驾驶员的手部是否离开了方向盘，若左手骨的三维坐标与右手骨的三维坐标均在所述的包络空间内，则驾驶员的手部未离开方向盘；否则驾驶员的手部离开了方向盘；Step 21. Determine whether the driver's hand has left the steering wheel according to whether the three-dimensional coordinates of the driver's left hand bone and the three-dimensional coordinates of the right hand bone are in the envelope space. If the three-dimensional coordinates of the left hand bone and the three-dimensional coordinates of the right hand bone are are all within the enveloped space, the driver's hands have not left the steering wheel; otherwise, the driver's hands have left the steering wheel;

步骤22、根据当前方向盘转动频率是否高于方向盘标准转向频率判断驾驶员是否有急转弯的危险驾驶行为，若当前方向盘转动频率高于方向盘标准转向频率，则驾驶员有急转弯的危险驾驶行为；否则，驾驶员没有急转弯的危险驾驶行为；Step 22. According to whether the current steering wheel rotation frequency is higher than the steering wheel standard steering frequency, it is judged whether the driver has a dangerous driving behavior of making a sharp turn. If the current steering wheel rotation frequency is higher than the steering wheel standard steering frequency, the driver has a sharp turning dangerous driving behavior; Otherwise, the driver has no dangerous driving behavior of sharp turns;

步骤23、根据驾驶员当前时刻肩部实时向量与所述肩部标准向量的夹角判断驾驶员当前是否有不正确坐姿，若当前时刻肩部实时向量与所述肩部标准向量的夹角大于肩部夹角阈值，则驾驶员当前坐姿不正确，否则驾驶员当前坐姿正确；Step 23, judge whether the driver currently has an incorrect sitting posture according to the included angle between the real-time shoulder vector of the driver and the standard shoulder vector at the current moment, if the included angle between the real-time shoulder vector and the standard shoulder vector at the current moment is greater than shoulder angle threshold, the driver's current sitting posture is incorrect, otherwise the driver's current sitting posture is correct;

步骤24、根据驾驶员当前时刻头颈部实时向量与所述头颈部标准向量的夹角判断驾驶员头部是否有不正常晃动，若当前时刻头颈部实时向量与所述头颈部标准向量的夹角大于颈部阈值，则驾驶员当前头部有不正常晃动；否则驾驶员当前头部未出现不正常晃动。Step 24. Determine whether the driver's head shakes abnormally according to the angle between the real-time vector of the head and neck at the current moment of the driver and the standard vector of the head and neck. If the included angle of the vector is greater than the neck threshold, the driver's current head shakes abnormally; otherwise, the driver's current head does not shake abnormally.

进一步地，所述的步骤22中当前T时刻方向盘转动频率fTfxp采用式X获得：Further, the steering wheel rotation frequency fTfxp at the current T moment in step 22 is obtained by using formula X:

Figure BDA0001894766580000051
Figure BDA0001894766580000051

其中，tT为当前T时刻的时间，tT-1为T-1时刻的时间，

Figure BDA0001894766580000052
为T-1时刻手骨向量，
Figure BDA0001894766580000053
为左手骨骼T-1时刻X轴向坐标，
Figure BDA0001894766580000054
为右手骨T-1时刻X轴向坐标；
Figure BDA0001894766580000055
为当前T时刻手骨向量，
Figure BDA0001894766580000056
为左手骨T时刻X轴向坐标，
Figure BDA0001894766580000057
为右手骨T时刻X轴向坐标。Among them, tT is the time at the current T moment, tT-1 is the time at T-1 moment,
Figure BDA0001894766580000052
is the hand bone vector at time T-1,
Figure BDA0001894766580000053
is the X-axis coordinate of the left hand bone at time T-1,
Figure BDA0001894766580000054
is the X-axis coordinate of the right hand bone at time T-1;
Figure BDA0001894766580000055
is the hand bone vector at the current time T,
Figure BDA0001894766580000056
is the X-axis coordinate of the left hand bone at time T,
Figure BDA0001894766580000057
is the X-axis coordinate of the right hand bone at time T.

进一步地，所述的步骤23中驾驶员当前T时刻肩部实时向量与所述肩部标准向量的夹角θ1T采用式XII获得：Further, in the step 23, the included angle θ1T between the real-time shoulder vector of the driver at the current moment T and the standard shoulder vector is obtained by formula XII:

Figure BDA0001894766580000058
Figure BDA0001894766580000058

其中，VJS为当前T时刻的肩部实时向量，VJS＝(XTrightshoulder-XTleftshoulder,YTrightshoulder-YTleftshoulder,ZTrightshoulder-ZTleftshoulder)，(XTleftshoulder,YTleftshoulder,ZTleftshoulder)为当前T时刻左肩关节的三维坐标，(XTrightshoulder,YTrightshoulder,ZTrightshoulder)为当前T时刻右肩关节的三维坐标。Wherein, VJS is the shoulder real-time vector at the current T moment, VJS = (XTrightshoulder -X Tleftshoulder , YTrightshoulder -Y Tleftshoulder , ZTrightshoulder -ZTleftshoulder ), (XTleftshoulder , YTleftshoulder , ZTleftshoulder ) is the three-dimensional coordinates of the left shoulder joint at the current T time, and (XTrightshoulder , YTrightshoulder , ZTrightshoulder ) is the three-dimensional coordinates of the right shoulder joint at the current T time.

进一步地，所述的步骤24中所述的驾驶员当前T时刻头颈部实时向量与所述头颈部标准向量的夹角θ2T采用式XIV获得：Further, the included angleθ2 T between the real-time vector of the head and neck of the driver at the current moment T and the standard vector of the head and neck described in step 24 is obtained by formula XIV:

Figure BDA0001894766580000059
Figure BDA0001894766580000059

其中，VNHS为当前T时刻的头颈部实时向量，VNHS＝(XThead-XTneck,YThead-YTneck,ZThead-ZTneck)，(XThead,YThead,ZThead)为当前T时刻头部骨骼的三维坐标，(XTneck,YTneck,ZTneck)为当前T时刻颈部骨骼的三维坐标。Among them, VNHS is the head and neck real-time vector at the current T moment, VNHS = (XThead -XTneck , YThead -YTneck , ZThead -ZTneck ), (XThead , YThead , ZThead ) are the three-dimensional coordinates of the head bones at the current T time, and (XTneck , YTneck , ZTneck ) are the three-dimensional coordinates of the neck bones at the current T time.

本发明与现有技术相比具有以下技术特点:Compared with the prior art, the present invention has the following technical characteristics:

1、本发明提供的驾驶员危险驾驶行为检测方法通过建立标准驾驶姿态模型，判断驾驶员行进过程中的姿态与标准驾驶姿态模型之间的偏差，判断驾驶员是否出现了危险驾驶的行为，检测过程无需对驾驶员进行干预，从而不会影响驾驶员的正常行驶，提高了检测结果的准确率；1. The driver’s dangerous driving behavior detection method provided by the present invention establishes a standard driving posture model, judges the deviation between the driver’s posture in the process of traveling and the standard driving posture model, and judges whether the driver has a dangerous driving behavior. The process does not need to intervene with the driver, so that it will not affect the driver's normal driving and improve the accuracy of the detection results;

2、本发明提供的驾驶员危险驾驶行为检测方法可以获得方向盘的各种参数，实现各种车辆的方向盘参数自适应检测，无需任何手动的参数输入。通过深度数据获取的车辆方向盘数据并对方向盘建模后，可以无需任何传感器和数据接口即可实现对驾驶行为进行检测，大大方便了驾驶行为的检测。2. The method for detecting the driver's dangerous driving behavior provided by the present invention can obtain various parameters of the steering wheel, and realize self-adaptive detection of steering wheel parameters of various vehicles without any manual parameter input. After the vehicle steering wheel data is obtained from the depth data and the steering wheel is modeled, the driving behavior can be detected without any sensors and data interfaces, which greatly facilitates the detection of driving behavior.

3、本发明提供的驾驶员危险驾驶行为检测方法可以快速开始对驾驶员的驾驶行为进行检测，且检测条件不受环境光线影响，很大程度上提升了基于视觉的驾驶员危险行为检测的可行性。3. The driver’s dangerous driving behavior detection method provided by the present invention can quickly start to detect the driver’s driving behavior, and the detection conditions are not affected by ambient light, which greatly improves the feasibility of the driver’s dangerous behavior detection based on vision. sex.

附图说明Description of drawings

图1为本发明提供的驾驶员危险驾驶行为检测方法的流程示意图；Fig. 1 is a schematic flow chart of the driver's dangerous driving behavior detection method provided by the present invention;

图2为本发明的一个实施例中提供的方向盘包络空间示意图；Fig. 2 is a schematic diagram of the envelope space of the steering wheel provided in one embodiment of the present invention;

图3为本发明的一个实施例中提供的方向盘投影在X轴以及Z轴方向示意图。Fig. 3 is a schematic diagram of the projection of the steering wheel provided in an embodiment of the present invention in the X-axis and Z-axis directions.

具体实施方式Detailed ways

驾驶员的标准姿态是指驾驶员双手把持方向盘时的姿势，当驾驶员保持标准姿态时，驾驶员双手之间的距离、驾驶员上臂和后臂之间的夹角、背部中心线与垂直于水平面方向的夹角以及头颈部中心线与垂直于水平面方向的夹角均保持在一定的范围内，因不同车辆驾驶室不同，方向盘角度和位置也不同，驾驶员的身高体型均不相同，因此通过定义驾驶员标准驾驶姿势范围，当满足此范围时，系统记录此时的驾驶员骨骼姿态作为标准姿态。The driver's standard posture refers to the posture when the driver holds the steering wheel with both hands. When the driver maintains the standard posture, the distance between the driver's hands, the angle between the driver's upper arm and rear arm, the centerline of the back and the vertical The angle between the horizontal plane direction and the angle between the centerline of the head and neck and the direction perpendicular to the horizontal plane is kept within a certain range. Because the cabs of different vehicles are different, the angle and position of the steering wheel are also different, and the height and shape of the drivers are not the same. Therefore, by defining the range of the driver's standard driving posture, when this range is met, the system records the driver's skeletal posture at this time as the standard posture.

例如，驾驶员双手之间的距离保持在肩部距离的0.5倍至1.5倍之间时，说明驾驶员双手距离符合标准姿态；驾驶员左上臂和左后臂之间的角度在90度至170度之间时，驾驶员右上臂和右后臂之间的角度在90度至170度之间时，说明驾驶员双臂姿态符合标准姿态；驾驶员坐下时背部中心线与垂直于水平面方向的夹角在0度到20度之间时，说明驾驶员坐姿符合标准姿态；驾驶员坐下时头颈部中心线与垂直于水平面方向的夹角在0度到10度之间时，说明驾驶员头部姿态符合标准姿态。For example, when the distance between the driver's hands is kept between 0.5 and 1.5 times the shoulder distance, it means that the distance between the driver's hands conforms to the standard posture; the angle between the driver's left upper arm and left rear arm is between 90 degrees and 170 degrees When the angle between the driver's right upper arm and the right rear arm is between 90 degrees and 170 degrees, it means that the driver's arms posture conforms to the standard posture; when the driver sits down, the back centerline and the direction perpendicular to the horizontal plane When the included angle between 0° and 20° is between 0° and 20°, it means that the driver’s sitting posture conforms to the standard posture; The driver's head posture conforms to the standard posture.

方向盘包络空间：包围环绕在方向盘外侧的空间，该空间能够将方向盘完全容纳。Steering wheel envelope space: the space surrounding the outer side of the steering wheel, which can completely accommodate the steering wheel.

Kinect摄像头：3D体感摄影头，可以捕捉三维空间中人体的运动姿态，在Kinect摄像头具有骨骼跟踪技术，可以捕获用户身体20个关节点的三维位置信息，即获得人体的骨骼姿态。Kinect camera: 3D somatosensory camera, which can capture the movement posture of the human body in three-dimensional space. The Kinect camera has bone tracking technology, which can capture the three-dimensional position information of 20 joint points of the user's body, that is, obtain the skeleton posture of the human body.

以下是发明人给出的具体实施例，以对本发明的技术方案进行进一步地说明。The following are specific examples given by the inventors to further illustrate the technical solutions of the present invention.

实施例一Embodiment one

如图1所示，本发明公开了一种驾驶员危险驾驶行为检测方法，所述的方法包括：As shown in Figure 1, the present invention discloses a method for detecting a driver's dangerous driving behavior, the method comprising:

步骤1、驾驶员双手握住方向盘的两侧并保持标准姿态时，采用Kinect摄像头采集驾驶员此时的骨骼姿态，建立标准姿态模型；Step 1. When the driver holds both sides of the steering wheel with both hands and maintains a standard posture, use the Kinect camera to collect the skeleton posture of the driver at this time, and establish a standard posture model;

本发明提出了一种使用Kinect深度摄像头提取驾驶员的姿态并分析驾驶员有无危险驾驶行为的方法。该方法无需让驾驶员穿戴各种测量传感器，也无需在车辆上安装大量传感器，真正实现无接触的驾驶员行为检测，不会对驾驶员的正常驾驶产生任何干扰，并且可以做到白天和夜间等低光照情况下对驾驶员行为进行检测。The invention proposes a method for extracting a driver's posture by using a Kinect depth camera and analyzing whether the driver has dangerous driving behavior. This method does not require the driver to wear various measurement sensors, nor does it need to install a large number of sensors on the vehicle, and truly realizes non-contact driver behavior detection without any interference with the driver's normal driving, and can be done during the day and at night Detect driver behavior in low-light conditions.

Microsoft公司的Kinect体感摄像头有三个镜头，中间的镜头是RGB彩色摄影机，用来采集彩色图像。左右两边镜头则分别为红外线发射器和红外线CMOS摄影机所构成的3D结构光深度感应器，用来采集深度数据，即场景中物体到摄像头的距离。The Kinect somatosensory camera of Microsoft Company has three lenses, and the lens in the middle is an RGB color camera for collecting color images. The left and right lenses are 3D structured light depth sensors composed of infrared emitters and infrared CMOS cameras, which are used to collect depth data, that is, the distance from objects in the scene to the camera.

在本方法开始时，首先需要初始化标准骨骼姿态、标准姿态模型、Kinect摄像头、OpenGL三维库以及OpenGL渲染环境。At the beginning of this method, it is first necessary to initialize the standard skeleton pose, the standard pose model, the Kinect camera, the OpenGL 3D library, and the OpenGL rendering environment.

在本实施例中，在对Kinect摄像头进行初始化时，使用NiTE库命令NiTE::initialize()完成。In this embodiment, the NiTE library command NiTE::initialize() is used to initialize the Kinect camera.

再对参数进行初始化，包括方向盘平面与地面的夹角Afxp，正常方向盘转向半圈需要的时间Tfxp，急转向检测灵敏度SRO，识别灵敏度为SA。Then initialize the parameters, including the angle Afxp between the steering wheel plane and the ground, the time Tfxp required for the normal steering wheel to turn half a circle, the sharp steering detection sensitivity SRO , and the recognition sensitivity SA .

在本实施例中，初始化Afxp＝30°，Tfxp＝4s，SRO＝1.0，SA＝1.0。In this embodiment, Afxp =30°, Tfxp =4s, SRO =1.0, SA =1.0 are initialized.

在开始让驾驶员进行姿态校准时，首先延时3秒，为了让驾驶员有充足的时间做出标准的驾驶姿势。在3秒过后，驾驶员的骨骼姿势被记录下来，骨骼姿态信息被保存入结构体BodyPoints中，获得标准骨骼姿态，包括：When starting to let the driver perform attitude calibration, first delay for 3 seconds, in order to allow the driver enough time to make a standard driving posture. After 3 seconds, the driver's skeletal pose is recorded, and the skeletal pose information is saved into the structure BodyPoints to obtain the standard skeletal pose, including:

步骤A、创建用户跟踪器对象UserTracker，并使用命令UserTracker::create()创建用户追踪器；Step A, create a user tracker object UserTracker, and use the command UserTracker::create() to create a user tracker;

步骤B、创建用户场景点云帧对象UserTrackerFrameRef，并使用命令UserTracker::readFrame(&UserTrackerFrameRef)读取点云帧；Step B. Create the user scene point cloud frame object UserTrackerFrameRef, and use the command UserTracker::readFrame(&UserTrackerFrameRef) to read the point cloud frame;

步骤C、使用命令UserTracker::startSkeletonTracking()从点云帧中追踪骨架；Step C, use the command UserTracker::startSkeletonTracking() to track the skeleton from the point cloud frame;

步骤D、使用命令UserTracker::getSkeleton()将得到的骨胳分解，得到驾驶员当前的骨骼姿态，记为标准骨骼姿态。Step D. Use the command UserTracker::getSkeleton() to decompose the obtained skeleton to obtain the current skeleton pose of the driver, which is recorded as the standard skeleton pose.

在本实施例中，驾驶员坐在驾驶座位上同时满足以下条件后，则说明驾驶员当前保持了标准姿态：In this embodiment, after the driver sits on the driver's seat and satisfies the following conditions at the same time, it means that the driver currently maintains a standard posture:

Figure BDA0001894766580000091
Figure BDA0001894766580000091

其中，驾驶员左手骨三维坐标(Xlefthand,Ylefthand,Zlefthand)，右手骨三维坐标(Xrighthand,Yrighthand,Zrighthand)；驾驶员左前臂的方向向量Vlfa＝(Xlefthand-Xleftelbow,Ylefthand-Yleftelbow,Zlefthand-Zleftelbow)，右前臂的方向向量Vrfa＝(Xrighthand-Xrightelbow,Yrighthand-Yrightelbow,Zrighthand-Zrightelbow)；驾驶员左后臂的方向向量Vlua＝(Xleftshoulder-Xleftelbow,Yleftshoulder-Yleftelbow,Zleftshoulder-Zleftelbow)，右后臂的方向向量Vrua＝(Xrightshoulder-Xrightelbow,Yrightshoulder-Yrightelbow,Zrightshoulder-Zrightelbow)其中，左肘关节三维坐标(Xleftelbow,Yleftelbow,Zleftelbow)，右肘关节三维坐标(Xrightelbow,Yrightelbow,Zrightelbow)，左肩关节三维坐标(Xleftshoulder,Yleftshoulder,Zleftshoulder)，右肩关节三维坐标(Xrightshoulder,Yrightshoulder,Zrightshoulder)；Among them, the three-dimensional coordinates of the driver's left hand bone (Xlefthand , Ylefthand , Zlefthand ), the three-dimensional coordinates of the right hand bone (Xrighthand , Yrighthand , Zrighthand ); the direction vector Vlfa of the driver's left forearm = (Xlefthand -Xleftelbow ,Ylefthand -Yleftelbow ,Zlefthand -Zleftelbow ), the direction vector of the right forearm Vrfa =(Xrighthand -Xrightelbow ,Yrighthand -Yrightelbow ,Zrighthand -Zrightelbow ); the direction vector of the driver's left rear arm Vlua =(Xleftshoulder -Xleftelbow ,Yleftshoulder -Yleftelbow ,Zleftshoulder -Zleftelbow ), the direction vector of the right rear arm Vrua =(Xrightshoulder -Xrightelbow ,Yrightshoulder -Yrightelbow ,Zrightshoulder -Zrightelbow ) Among them, the three-dimensional coordinates of the left elbow joint (Xleftelbow , Yleftelbow , Zleftelbow ), the three-dimensional coordinates of the right elbow joint (Xrightelbow , Yrightelbow , Zrightelbow ), the three-dimensional coordinates of the left shoulder joint (Xleftshoulder , Yleftshoulder , Zleftshoulder ), Three-dimensional coordinates of the right shoulder joint (Xrightshoulder , Yrightshoulder , Zrightshoulder );

驾驶员坐姿向量Vzz＝(Xhc-Xneck,Yhc-Yneck,Zhc-Zneck)，头颈部向量Vhn＝(Xhead-Xneck,Yhead-Yneck,Zhead-Zneck)；Driver sitting posture vector Vzz =(Xhc -Xneck ,Yhc -Yneck ,Zhc -Zneck ), head and neck vector Vhn =(Xhead -Xneck ,Yhead -Yneck ,Zhead - Zneck );

其中，头部三维坐标(Xhead,Yhead,Zhead)，颈部三维坐标(Xneck,Yneck,Zneck)，髋中心点三维坐标为(Xhc,Yhc,Zhc)。Wherein, the three-dimensional coordinates of the head (Xhead , Yhead , Zhead ), the three-dimensional coordinates of the neck (Xneck , Yneck , Zneck ), and the three-dimensional coordinates of the hip center point are (Xhc , Yhc , Zhc ).

在本实施例中，驾驶员的左肩坐标为(Xleftshoulder,Yleftshoulder,Zleftshoulder)＝(-5.0,0.0,0.0)，右肩坐标为(Xrighthoulder,Yrightshoulder,Zrightshoulder)＝(5.0,0.0,0.0)；左肘坐标为(Xleftelbow,Yleftelbow,Zleftelbow)＝(-5.0,-1.0,-2.5)，右肘坐标为(Xrightelbow,Yrightelbow,Zrightelbow)＝(5.0,-1.0,-2.5)；左手骨坐标为(Xlefthand,Ylefthand,Zlefthand)＝(-5.0,-1.0,-5.0)，右手骨坐标为(Xrighthand,Yrighthand,Zrighthand)＝(5.0,-1.0,-5.0)；头部坐标为(Xhead,Yhead,Zhead)＝(0.0,2.0,0.0)，颈部坐标为(Xneck,Yneck,Zneck)＝(0.0,1.0,0.0)，髋中心点三维坐标为(Xhc,Yhc,Zhc)＝(0.0,-4.0,0.0)。In this embodiment, the driver's left shoulder coordinates are (Xleftshoulder , Yleftshoulder , Zleftshoulder )=(-5.0,0.0,0.0), and the right shoulder coordinates are (Xrightshoulder ,Yrightshoulder ,Zrightshoulder )=(5.0, 0.0,0.0); left elbow coordinates are (Xleftelbow ,Yleftelbow ,Zleftelbow )=(-5.0,-1.0,-2.5), right elbow coordinates are (Xrightelbow ,Yrightelbow ,Zrightelbow )=(5.0,- 1.0,-2.5); the left hand bone coordinates are (Xlefthand ,Ylefthand ,Zlefthand )=(-5.0,-1.0,-5.0), and the right hand bone coordinates are (Xrighthand ,Yrighthand ,Zrighthand )=(5.0, -1.0,-5.0); the head coordinates are (Xhead ,Yhead ,Zhead )=(0.0,2.0,0.0), the neck coordinates are (Xneck ,Yneck ,Zneck )=(0.0,1.0, 0.0), the three-dimensional coordinates of the hip center point are (Xhc , Yhc , Zhc )=(0.0,-4.0,0.0).

因为|Xleftshoulder-Xrighthoulder|＝10.0，|Xlefthand-Xrighthand|＝10.0，则符合范围0.5*|Xleftshoulder-Xrighthoulder|<|Xlefthand-Xrighthand|<1.5*|Xleftshoulder-Xrighthoulder|，因此认为驾驶员双手距离符合标准驾驶姿势。Because |Xleftshoulder -Xrightshoulder |＝10.0, |Xlefthand -Xrighthand |＝10.0, it fits the range 0.5*|Xleftshoulder -Xrightshoulder |<|Xlefthand -Xrighthand |<1.5*|Xleftshoulder -Xrighthander |, so the distance between the driver's hands is considered to be in line with the standard driving posture.

驾驶员左前臂方向向量Vlfa＝(0.0,0.0,-2.5)，可以得到右前臂的方向向量Vrfa＝(0.0,0.0,-2.5)，可以得到驾驶员左后臂的方向向量Vlua＝(0.0,1.0,2.5)，可以得到右后臂的方向向量Vrua＝(0.0,1.0,2.5)，那么可以计算出左上臂和左后臂的夹角θla＝158.2°，同理可以计算出右上臂和右后臂的夹角θra＝158.2°，符合90°<θra＝θra<170°，因此可以判断驾驶员的双臂姿态符合标准驾驶姿势。The driver's left forearm direction vector Vlfa =(0.0,0.0,-2.5), the right forearm direction vector Vrfa =(0.0,0.0,-2.5), the driver's left rear arm direction vector Vlua = (0.0, 1.0, 2.5), the direction vector Vrua of the right rear arm can be obtained = (0.0, 1.0, 2.5), then the angle θla = 158.2° between the left upper arm and the left rear arm can be calculated, and similarly can be calculated The included angle θra =158.2° between the right upper arm and the right rear arm conforms to 90°<θra =θra <170°, so it can be judged that the driver's arms posture conforms to the standard driving posture.

通过髋中心点和颈部三维坐标可以得到驾驶员坐姿向量Vzz＝(0.0,-5.0,0.0)，则驾驶员坐姿向量与车辆Y轴夹角可以计算为θzz＝0，则符合0°≤θzz≤20°，可以认为驾驶员坐姿符合标准驾驶姿势。The driver's sitting posture vector Vzz = (0.0, -5.0, 0.0) can be obtained through the three-dimensional coordinates of the hip center point and the neck, then the angle between the driver's sitting posture vector and the Y-axis of the vehicle can be calculated as θzz = 0, which corresponds to 0° ≤θzz ≤20°, it can be considered that the driver’s sitting posture conforms to the standard driving posture.

通过颈部和头部的三维坐标可以得到头颈部向量Vhn＝(0.0,1.0,0.0)，则头颈部向量与车辆Y轴夹角可以计算为θhn＝0，则符合0°≤θhn≤10°可以认为驾驶员头部姿势符合标准驾驶姿势。The head and neck vector Vhn = (0.0, 1.0, 0.0) can be obtained through the three-dimensional coordinates of the neck and head, then the angle between the head and neck vector and the Y-axis of the vehicle can be calculated as θhn = 0, which meets 0°≤ When θhn ≤ 10°, it can be considered that the driver's head posture conforms to the standard driving posture.

综上所述，驾驶员双手距离、双臂姿态、坐姿、头部姿势均符合标准姿态，系统认为驾驶员已经做出标准姿态，系统记录此时的驾驶员骨骼姿态的数值，建立标准姿态模型。To sum up, the distance between the driver's hands, the posture of the arms, the sitting posture, and the head posture all conform to the standard posture. The system believes that the driver has made the standard posture. The system records the value of the driver's bone posture at this time and establishes the standard posture model. .

在本步骤中，驾驶员双手握住方向盘的两侧并保持标准姿态时，采用Kinect摄像头采集驾驶员此时的骨骼姿态包括：左手骨三维坐标(Xlefthand,Ylefthand,Zlefthand)，右手骨三维坐标(Xrighthand,Yrighthand,Zrighthand)，左肩关节X轴坐标(Xleftshoulder,Yleftshoulder,Zleftshoulder)，右肩关节X轴坐标(Xrightshoulder,Yrightshoulder,Zrightshoulder)，头部三维坐标(Xhead,Yhead,Zhead)，颈部三维坐标(Xneck,Yneck,Zneck)。In this step, when the driver holds both sides of the steering wheel with both hands and maintains a standard posture, the Kinect camera is used to collect the skeletal posture of the driver at this time, including: the three-dimensional coordinates of the left hand bone (Xlefthand , Ylefthand , Zlefthand ), right hand bone Three-dimensional coordinates (Xrighthand , Yrighthand , Zrighthand ), X-axis coordinates of the left shoulder joint (Xleftshoulder , Yleftshoulder , Zleftshoulder ), X-axis coordinates of the right shoulder joint (Xrightshoulder , Yrightshoulder , Zrightshoulder ), three-dimensional coordinates of the head (Xhead , Yhead , Zhead ), the three-dimensional coordinates of the neck (Xneck , Yneck , Zneck ).

其中，建立标准姿态模型，包括：Among them, a standard posture model is established, including:

在本方案中，为了快速检测手骨坐标是否超出方向盘包络域范围，假设方向盘包络空间为与空间坐标轴平行的六面体，如图2所示。In this scheme, in order to quickly detect whether the coordinates of the hand bones exceed the range of the envelope domain of the steering wheel, it is assumed that the envelope space of the steering wheel is a hexahedron parallel to the spatial coordinate axis, as shown in Figure 2.

步骤11、建立如式I所示的方向盘标准包络空间，所述方向盘标准包括空间在X轴、Y轴以及Z轴上的坐标的单位均为单位长度：Step 11, establish the steering wheel standard envelope space shown in formula I, the steering wheel standard includes the coordinates of the space on the X-axis, Y-axis and Z-axis in units of unit length:

Figure BDA0001894766580000121
Figure BDA0001894766580000121

其中，(Xlefthand,Ylefthand,Zlefthand)为驾驶员在保持标准姿态时的左手骨三维坐标，(Xrighthand,Yrighthand,Zrighthand)为驾驶员在保持标准姿态时的右手骨三维坐标，SWxmin为方向盘标准包络空间X轴向上坐标最小值，SWxmax为方向盘标准包络空间X轴向上坐标最大值，SWymin为方向盘标准包络空间Y轴向上坐标最小值，SWymax为方向盘标准包络空间Y轴向上坐标最大值，SWzmin为方向盘标准包络空间Z轴向上坐标最小值，SWzmax为方向盘标准包络空间Z轴向上坐标最大值，Lfxp为方向盘的长度在方向盘标准包络空间中X轴向上的投影距离，Hfxp为方向盘的高在方向盘标准包络空间中Y轴向上的投影距离，Wfxp为方向盘的宽度在方向盘标准包络空间中Z轴向上的投影距离；Among them, (Xlefthand , Ylefthand , Zlefthand ) is the three-dimensional coordinates of the driver's left hand bone when maintaining a standard posture, (Xrighthand , Yrighthand , Zrighthand ) is the three-dimensional coordinates of the driver's right hand bone when maintaining a standard posture, SWxmin is the minimum value of the upper coordinate of the X-axis in the standard envelope space of the steering wheel, SWxmax is the maximum value of the upper coordinate of the X-axis in the standard envelope space of the steering wheel, SWymin is the minimum value of the upper coordinate of the Y-axis in the standard envelope space of the steering wheel, and SWymax is the maximum value of the upper coordinate of the Y axis in the standard envelope space of the steering wheel, SWzmin is the minimum value of the upper coordinate of the Z axis in the standard envelope space of the steering wheel, SWzmax is the maximum value of the upper coordinate of the Z axis in the standard envelope space of the steering wheel, and Lfxp is the maximum value of the upper coordinate of the Z axis in the standard envelope space of the steering wheel. The projection distance of the length of the length of the steering wheel on the X axis in the standard envelope space of the steering wheel, Hfxp is the projection distance of the height of the steering wheel on the Y axis in the standard envelope space of the steering wheel, Wfxp is the width of the steering wheel in the standard envelope space of the steering wheel The projection distance on the middle Z axis;

在本步骤中，通过式II计算方向盘长Lfxp在X轴向上的投影距离，单位为单位长度：In this step, the projection distance of the steering wheel length Lfxp on the X-axis is calculated by formula II, and the unit is unit length:

Lfxp＝|Xrighthand-Xlefthand|×SA 式IILfxp ＝|Xrighthand -Xlefthand |×SA formula II

通过式III计算方向盘宽Wfxp在Z轴向上的投影距离，单位为单位长度：Calculate the projected distance of the steering wheel width Wfxp on the Z axis through formula III, and the unit is unit length:

Wfxp＝Lfxp·cos(Afxp) 式IIIWfxp ＝Lfxp ·cos(Afxp ) Formula III

通过式IV计算方向盘高Hfxp在Y轴向上的投影距离，单位为单位长度：Calculate the projected distance of the steering wheel height Hfxp on the Y axis through formula IV, the unit is unit length:

Hfxp＝Lfxp·sin(Afxp) 式IVHfxp ＝Lfxp ·sin(Afxp ) Formula IV

在本实施例中，如图3所示，方向盘的长度在方向盘标准包络空间中X轴向上的投影距离Lfxp＝2.0个单位长度，方向盘的宽度在方向盘标准包络空间中Y轴向上的投影距离Wfxp＝2.0个单位长度，方向盘的高在方向盘标准包络空间中Z轴向上的投影距离Hfxp＝1.0个单位长度，因此方向盘标准包络空间的三维坐标为：In the present embodiment, as shown in Figure 3, the projection distanceLfxp =2.0 unit lengths of the length of the steering wheel on the X-axis in the standard envelope space of the steering wheel, and the width of the steering wheel in the Y-axis in the standard envelope space of the steering wheel The projected distance Wfxp = 2.0 unit lengths, and the projection distance Hfxp = 1.0 unit lengths of the height of the steering wheel on the Z axis in the standard envelope space of the steering wheel, so the three-dimensional coordinates of the standard envelope space of the steering wheel are:

Figure BDA0001894766580000141
Figure BDA0001894766580000141

步骤12、采用式V计算方向盘标准转向频率ffxp：Step 12. Calculate the standard steering frequency ffxp of the steering wheel using formula V:

Figure BDA0001894766580000142
Figure BDA0001894766580000142

其中，Rfxp为方向盘的半径在方向盘标准包络空间中X轴向上的投影距离，

Figure BDA0001894766580000143
SRO为急转向检测灵敏度，SRO>0,Tfxp为方向盘正常转向半圈需要的时间，单位为s；Among them, Rfxp is the projection distance of the radius of the steering wheel on the X-axis in the standard envelope space of the steering wheel,
Figure BDA0001894766580000143
SRO is the sharp steering detection sensitivity, SRO >0, Tfxp is the time required for the steering wheel to turn half a circle normally, the unit is s;

在本实施例中，方向盘的半径在方向盘标准包络空间X轴上的投影距离Rfxp＝2.0个单位长度，急转向检测灵敏度SWRoSensi为1.0，方向盘正常转向半圈需要的时间Tfxp＝4s，则方向盘标准转向频率可以计算得到ffxp＝0.5。In this embodiment, the projection distance Rfxp of the radius of the steering wheel on the X-axis of the standard envelope space of the steering wheel is 2.0 unit lengths, the sharp steering detection sensitivity SWRoSensi is 1.0, and the time required for the steering wheel to turn half a circle normally Tfxp =4s , then the standard steering frequency of the steering wheel can be calculated to obtain ffxp =0.5.

步骤13、采用式VI计算驾驶员肩部标准向量VJB：Step 13. Calculate the driver's shoulder standard vector VJB using formula VI:

Figure BDA0001894766580000144
Figure BDA0001894766580000144

其中，(Xleftshoulder,Yleftshoulder,Zleftshoulder)为驾驶员在保持标准姿态时的左肩关节三维坐标，(Xrightshoulder,Yrightshoulder,Zrightshoulder)为驾驶员在保持标准姿态时的右肩关节三维坐标；Among them, (Xleftshoulder , Yleftshoulder , Zleftshoulder ) is the three-dimensional coordinates of the driver's left shoulder joint when maintaining a standard posture, (Xrightshoulder , Yrightshoulder , Zrightshoulder ) is the three-dimensional coordinates of the driver's right shoulder joint when maintaining a standard posture ;

在本实施例中，Xrightshoulder＝5.0，Xleftshoulder＝-5.0，Yrightshoulder＝Yleftshoulder＝0.0，Zrightshoulder＝Zleftshoulder＝0.0，则肩部标准向量VJB＝(10.0,0.0,0.0)。In this embodiment, Xrightshoulder =5.0, Xleftshoulder =-5.0, Yrightshoulder =Yleftshoulder =0.0, Zrightshoulder =Zleftshoulder =0.0, then the shoulder standard vector VJB =(10.0,0.0,0.0).

步骤14、采用式VII计算驾驶员头颈部标准向量VNH：Step 14. Calculate the standard vector VNH of the driver's head and neck using formula VII:

VNH＝(Xhead-Xneck,Yhead-Yneck,Zhead-Zneck) 式VIIVNH ＝(Xhead -Xneck ,Yhead -Yneck ,Zhead -Zneck ) formula VII

其中，(Xhead,Yhead,Zhead)为驾驶员在保持标准姿态时的头部三维坐标，(Xneck,Yneck,Zneck)为驾驶员在保持标准姿态时的颈部三维坐标；Among them, (Xhead , Yhead , Zhead ) are the three-dimensional coordinates of the driver's head when maintaining a standard posture, and (Xneck , Yneck , Zneck ) are the three-dimensional coordinates of the driver's neck when maintaining a standard posture;

在本步骤中，Xhead＝0.0，Xneck＝0.0，Yhead＝5.0，Yneck＝0.0，Zhead＝0.0，Zneck＝0.0，那么头颈部标准向量VNH＝(0.0,5.0,0.0)。In this step, Xhead =0.0, Xneck =0.0, Yhead =5.0, Yneck =0.0, Zhead =0.0, Zneck =0.0, then the head and neck standard vector VNH =(0.0,5.0,0.0 ).

步骤2、在驾驶员驾驶过程中重复执行本步骤，检测驾驶员在驾驶过程中是否具有危险驾驶行为：Step 2. Repeat this step during the driving process of the driver to detect whether the driver has dangerous driving behavior during the driving process:

利用Kinect摄像头采集驾驶员的骨骼姿态，与所述的标准姿态模型进行对比，检测驾驶员是否具有危险驾驶行为，包括：Use the Kinect camera to collect the driver's skeletal posture, compare it with the standard posture model, and detect whether the driver has dangerous driving behavior, including:

步骤21、根据驾驶员左手骨的三维坐标与右手骨的三维坐标是否均在所述的包络空间内判断驾驶员的手部是否离开了方向盘，若左手骨的三维坐标与右手骨的三维坐标均在所述的包络空间内，则驾驶员的手部未离开方向盘；否则驾驶员的手部离开了方向盘；Step 21. Determine whether the driver's hand has left the steering wheel according to whether the three-dimensional coordinates of the driver's left hand bone and the three-dimensional coordinates of the right hand bone are in the envelope space. If the three-dimensional coordinates of the left hand bone and the three-dimensional coordinates of the right hand bone are are all within the enveloped space, the driver's hands have not left the steering wheel; otherwise, the driver's hands have left the steering wheel;

在本实施例中，当前T时刻左手骨三维坐标为(XTlefthand,YTlefthand,ZTlefthand)＝(-15,0,0)，当前T时刻右手骨三维坐标(XTrighthand,YTrighthand,ZTrighthand)＝(9,0,0)，则与标准包络空间进行对比，驾驶员在T时刻左手离开了方向盘，右手未离开方向盘，当前T+1时刻左手骨三维坐标为(XT+1lefthand,YT+1lefthand,ZT+1lefthand)＝(0,5,0)，当前T+1时刻右手骨三维坐标为(XT+1righthand,YT+1righthand,ZT+1righthand)＝(0,-15,0)，则与标准包络空间进行对比，驾驶员在T+1时刻左手未离开了方向盘，右手离开了方向盘。In this embodiment, the three-dimensional coordinates of the left hand bone at the current T moment are (XTlefthand , YTlefthand , ZTlefthand )=(-15,0,0), and the three-dimensional coordinates of the right hand bone at the current T moment (XTrighthand ,YTrighthand , ZTrighthand )=(9,0,0), then compared with the standard envelope space, the driver left the steering wheel with his left hand at time T, but his right hand did not leave the steering wheel. The three-dimensional coordinates of the left hand bone at the current time T+1 are (XT+1lefthand ,YT+1lefthand ,ZT+1lefthand )=(0,5,0), the three-dimensional coordinates of the right hand bone at the current T+1 moment are (XT+1righthand ,YT+1righthand ,ZT+1righthand )=(0,-15,0), then compared with the standard envelope space, the driver did not leave the steering wheel with his left hand at time T+1, but his right hand left the steering wheel.

步骤22、根据当前方向盘转动频率是否高于方向盘标准转向频率判断驾驶员是否有急转弯的危险驾驶行为，若当前方向盘转动频率高于方向盘标准转向频率，则驾驶员有急转弯的危险驾驶行为；否则，驾驶员没有急转弯的危险驾驶行为；Step 22. According to whether the current steering wheel rotation frequency is higher than the steering wheel standard steering frequency, it is judged whether the driver has a dangerous driving behavior of making a sharp turn. If the current steering wheel rotation frequency is higher than the steering wheel standard steering frequency, the driver has a sharp turning dangerous driving behavior; Otherwise, the driver has no dangerous driving behavior of sharp turns;

可选地，所述的步骤22中当前T时刻方向盘转动频率fTfxp采用式X获得：Optionally, the steering wheel rotation frequency fTfxp at the current T moment in step 22 is obtained by using formula X:

Figure BDA0001894766580000161
Figure BDA0001894766580000161

其中，tT为当前T时刻的时间，tT-1为T-1时刻的时间，

Figure BDA0001894766580000162
为T-1时刻手骨向量，
Figure BDA0001894766580000163
为左手骨骼T-1时刻X轴向坐标，
Figure BDA0001894766580000164
为右手骨T-1时刻X轴向坐标；
Figure BDA0001894766580000165
为当前T时刻手骨向量，
Figure BDA0001894766580000166
为左手骨T时刻X轴向坐标，
Figure BDA0001894766580000167
为右手骨T时刻X轴向坐标。Among them, tT is the time at the current T moment, tT-1 is the time at T-1 moment,
Figure BDA0001894766580000162
is the hand bone vector at time T-1,
Figure BDA0001894766580000163
is the X-axis coordinate of the left hand bone at time T-1,
Figure BDA0001894766580000164
is the X-axis coordinate of the right hand bone at time T-1;
Figure BDA0001894766580000165
is the hand bone vector at the current time T,
Figure BDA0001894766580000166
is the X-axis coordinate of the left hand bone at time T,
Figure BDA0001894766580000167
is the X-axis coordinate of the right hand bone at time T.

因此，所述的步骤22包括：Therefore, said step 22 includes:

步骤221、采用式VIII生成T-1时刻手骨向量

Figure BDA0001894766580000168
以及T时刻手骨向量
Figure BDA0001894766580000169
Step 221, using formula VIII to generate hand bone vector at time T-1
Figure BDA0001894766580000168
and hand bone vector at time T
Figure BDA0001894766580000169

Figure BDA00018947665800001610
Figure BDA00018947665800001610

其中，

Figure BDA00018947665800001611
为左手骨T时刻X轴向坐标，
Figure BDA00018947665800001612
为右手骨T时刻X轴向坐标，
Figure BDA00018947665800001613
为左手骨骼T-1时刻X轴向坐标，
Figure BDA00018947665800001614
为右手骨T-1时刻X轴向坐标；in,
Figure BDA00018947665800001611
is the X-axis coordinate of the left hand bone at time T,
Figure BDA00018947665800001612
is the X-axis coordinate of the right hand bone at time T,
Figure BDA00018947665800001613
is the X-axis coordinate of the left hand bone at time T-1,
Figure BDA00018947665800001614
is the X-axis coordinate of the right hand bone at time T-1;

在本实施例中，

Figure BDA0001894766580000171
则
Figure BDA0001894766580000172
下一个时刻左右手骨坐标为：
Figure BDA0001894766580000173
则
Figure BDA0001894766580000174
In this example,
Figure BDA0001894766580000171
but
Figure BDA0001894766580000172
The coordinates of the left and right hand bones at the next moment are:
Figure BDA0001894766580000173
but
Figure BDA0001894766580000174

步骤222、采用式IX判断驾驶员当前T时刻是否有转弯动作：Step 222, use formula IX to judge whether the driver has a turning action at the current T moment:

Figure BDA0001894766580000175
Figure BDA0001894766580000175

若式IX成立，则说明驾驶员有转弯动作，执行步骤223；否则说明驾驶员没有转弯动作，执行步骤23；If formula IX is established, it means that the driver has a turning action, and step 223 is performed; otherwise, the driver has no turning action, and step 23 is performed;

在本实施例中，在上一个步骤中计算出来的

Figure BDA0001894766580000176
Rfxp＝2.0，急转向检测灵敏度SWRoSensi为1.0，那么
Figure BDA0001894766580000177
Figure BDA0001894766580000178
则上式成立，因此可以检测出驾驶员在当前时刻有转弯动作。In this example, calculated in the previous step
Figure BDA0001894766580000176
Rfxp = 2.0, the sharp steering detection sensitivity SWRoSensi is 1.0, then
Figure BDA0001894766580000177
Figure BDA0001894766580000178
Then the above formula is established, so it can be detected that the driver has a turning action at the current moment.

步骤223、采用式X判断驾驶员当前T时刻是否有急转弯的危险驾驶行为：Step 223, using formula X to judge whether the driver has a dangerous driving behavior of making a sharp turn at the current time T:

Figure BDA0001894766580000179
Figure BDA0001894766580000179

其中，tT为T时刻的时间，tT-1为T-1时刻的时间；Wherein, tT is the time at time T, and tT-1 is the time at time T-1;

若式X成立，则说明驾驶员有急转弯的危险驾驶行为；否则说明驾驶员没有急转弯的危险驾驶行为。If formula X is established, it means that the driver has dangerous driving behavior of sharp turn; otherwise, it means that the driver does not have dangerous driving behavior of sharp turn.

在本实施例中，在以上步骤中得到了

Figure BDA00018947665800001710
tT-1＝1s,tT＝1.5s,则
Figure BDA00018947665800001711
大于步骤12中预设定的ffxp＝0.5，因此可以判断出在T时刻驾驶员有急转弯动作。In this example, obtained in the above steps
Figure BDA00018947665800001710
tT-1 ＝1s, tT ＝1.5s, then
Figure BDA00018947665800001711
is greater than ffxp =0.5 preset in step 12, so it can be judged that the driver makes a sharp turn at time T.

步骤23、根据驾驶员当前时刻肩部实时向量与所述肩部标准向量的夹角判断驾驶员当前是否有不正确坐姿，若当前时刻肩部实时向量与所述肩部标准向量的夹角大于肩部夹角阈值，则驾驶员当前坐姿不正确，否则驾驶员当前坐姿正确；Step 23, judge whether the driver currently has an incorrect sitting posture according to the included angle between the real-time shoulder vector of the driver and the standard shoulder vector at the current moment, if the included angle between the real-time shoulder vector and the standard shoulder vector at the current moment is greater than shoulder angle threshold, the driver's current sitting posture is incorrect, otherwise the driver's current sitting posture is correct;

可选地，所述的步骤23中驾驶员当前T时刻肩部实时向量与所述肩部标准向量的夹角θ1T采用式XII获得：Optionally, in the step 23, the included angle θ1T between the real-time shoulder vector of the driver at the current moment T and the shoulder standard vector is obtained by formula XII:

Figure BDA0001894766580000181
Figure BDA0001894766580000181

其中，VJS为当前T时刻的肩部实时向量，VJS＝(XTrightshoulder-XTleftshoulder,YTrightshoulder-YTleftshoulder,ZTrightshoulder-ZTleftshoulder)，(XTleftshoulder,YTleftshoulder,ZTleftshoulder)为当前T时刻左肩关节的三维坐标，(XTrightshoulder,YTrightshoulder,ZTrightshoulder)为当前T时刻右肩关节的三维坐标。Wherein, VJS is the shoulder real-time vector at the current T moment, VJS = (XTrightshoulder -X Tleftshoulder , YTrightshoulder -Y Tleftshoulder , ZTrightshoulder -ZTleftshoulder ), (XTleftshoulder , YTleftshoulder , ZTleftshoulder ) is the three-dimensional coordinates of the left shoulder joint at the current T time, and (XTrightshoulder , YTrightshoulder , ZTrightshoulder ) is the three-dimensional coordinates of the right shoulder joint at the current T time.

因此，所述的步骤23包括：Therefore, said step 23 includes:

步骤231、采用式XI计算当前T时刻的肩部实时向量VJS：Step 231. Calculate the shoulder real-time vector VJS at the current moment T by using formula XI:

VJS＝(XTrightshoulder-XTleftshoulder,YTrightshoulder-YTleftshoulder,ZTrightshoulder-ZTleftshoulder) 式XIVJS ＝(XTrightshoulder -XTleftshoulder ,YTrightshoulder -YTleftshoulder ,ZTrightshoulder -ZTleftshoulder ) Formula XI

其中，(XTleftshoulder,YTleftshoulder,ZTleftshoulder)为当前T时刻左肩关节的三维坐标，(XTrightshoulder,YTrightshoulder,ZTrightshoulder)为当前T时刻右肩关节的三维坐标；Among them, (XTleftshoulder , YTleftshoulder , ZTleftshoulder ) is the three-dimensional coordinates of the left shoulder joint at the current T moment, (XTrightshoulder , YTrightshoulder , ZTrightshoulder ) is the three-dimensional coordinates of the right shoulder joint at the current T moment;

在本实施例中，左肩坐标为(XTleftshoulder,YTleftshoulder,ZTleftshoulder)＝(-5,0,-2)，右肩坐标为(XTrightshoulder,YTrightshoulder,ZTrightshoulder)＝(5,0,2)，则可以计算出向量VJS＝(10,0,4)。In this embodiment, the coordinates of the left shoulder are (XTleftshoulder , YTleftshoulder , ZTleftshoulder )=(-5,0,-2), and the coordinates of the right shoulder are (XTrightshoulder , YTrightshoulder , ZTrightshoulder ) =(5,0,2), then the vector VJS =(10,0,4) can be calculated.

步骤232、采用式XII计算当前T时刻的肩部夹角θ1T：Step 232, using Formula XII to calculate the shoulder angle θ1T at the current moment T:

Figure BDA0001894766580000191
Figure BDA0001894766580000191

在本实施例中，在步骤13中已经得到了肩部标准向量VJB＝(10,0,0)。在上一步骤中也得到了实时向量VJS＝(10,0,4)，那么可以得到T时刻肩部夹角θ1T＝21.8°。In this embodiment, the shoulder standard vector VJB =(10,0,0) has been obtained in step 13 . In the previous step, the real-time vector VJS =(10,0,4) is also obtained, then the shoulder angle θ1T =21.8° at time T can be obtained.

步骤233、判断当前T时刻肩部夹角θ1T是否大于肩部夹角阈值，若当前T时刻肩部夹角θ1T大于肩部夹角阈值，则说明驾驶员当前坐姿不正确；否则，驾驶员当前坐姿正确。Step 233. Determine whether the shoulder angle θ1T is greater than the shoulder angle threshold at the current T time. If the shoulder angle θ1T is greater than the shoulder angle threshold at the current T time, it means that the driver's current sitting posture is incorrect; otherwise , the driver's current sitting posture is correct.

在本实施例中，定义肩部夹角阈值为20°，在上一步中判断出驾驶员的当前T时刻肩部转动夹角为21.8°，那么可以判断此时驾驶员当前T时刻坐姿不正确。In this embodiment, the shoulder angle threshold is defined as 20°. In the previous step, it is judged that the driver’s current T moment shoulder rotation angle is 21.8°, so it can be judged that the driver’s current T moment sitting posture is incorrect. .

步骤24、根据驾驶员当前时刻头颈部实时向量与所述头颈部标准向量的夹角判断驾驶员头部是否有不正常晃动，若当前时刻头颈部实时向量与所述头颈部标准向量的夹角大于颈部阈值，则驾驶员当前头部有不正常晃动；否则驾驶员当前头部未出现不正常晃动。Step 24. Determine whether the driver's head shakes abnormally according to the angle between the real-time vector of the head and neck at the current moment of the driver and the standard vector of the head and neck. If the included angle of the vector is greater than the neck threshold, the driver's current head shakes abnormally; otherwise, the driver's current head does not shake abnormally.

可选地，所述的步骤24中所述的驾驶员当前T时刻头颈部实时向量与所述头颈部标准向量的夹角θ2T采用式XIV获得：Optionally, the included angle θ2 T between the real-time vector of the head and neck of the driver at the current time T and the standard vector of the head and neck described in step 24 is obtained by formulaXIV :

Figure BDA0001894766580000192
Figure BDA0001894766580000192

其中，VNHS为当前T时刻的头颈部实时向量，VNHS＝(XThead-XTneck,YThead-YTneck,ZThead-ZTneck)，(XThead,YThead,ZThead)为当前T时刻头部骨骼的三维坐标，(XTneck,YTneck,ZTneck)为当前T时刻颈部骨骼的三维坐标。Among them, VNHS is the head and neck real-time vector at the current T moment, VNHS = (XThead -XTneck , YThead -YTneck , ZThead -ZTneck ), (XThead , YThead , ZThead ) are the three-dimensional coordinates of the head bones at the current T time, and (XTneck , YTneck , ZTneck ) are the three-dimensional coordinates of the neck bones at the current T time.

因此，所述的步骤24包括：Therefore, said step 24 includes:

步骤241、采用式XIII计算当前时刻T时刻头颈部实时向量VJS：Step 241: Calculate the head and neck real-time vector VJS at the current moment T by using Formula XIII:

VNHS＝(XThead-XTneck,YThead-YTneck,ZThead-ZTneck) 式XIIIVNHS ＝(XThead -XTneck , YThead -YTneck , ZThead -ZTneck ) Formula XIII

其中，(XThead,YThead,ZThead)为当前T时刻头部骨骼的三维坐标，(XTneck,YTneck,ZTneck)为当前T时刻颈部骨骼的三维坐标；Among them, (XThead , YThead , ZThead ) is the three-dimensional coordinates of the head bones at the current T time, and (XTneck , YTneck , ZTneck ) is the three-dimensional coordinates of the neck bones at the current T time;

在本实施例中，在当前T时刻头部骨骼的三维坐标(XThead,YThead,ZThead)＝(-5,5,0)，当前T时刻颈部骨骼的三维坐标(XTneck,YTneck,ZTneck)＝(0,0,0)，则当前T时刻头颈部实时向量VNHS＝(-5,5,0)。In this embodiment, at the current T moment, the three-dimensional coordinates of the head bones (XThead , YThead , ZThead )=(-5,5,0), and the three-dimensional coordinates of the neck bones at the current T moment (XTneck , YTneck , ZTneck )=(0,0,0), then the head and neck real-time vector VNHS =(-5,5,0) at the current T moment.

步骤242、采用式XIV计算当前T时刻头颈部夹角θ2T：Step 242. Calculate the head and neck angle θ2T at the current moment T by using formula XIV:

Figure BDA0001894766580000201
Figure BDA0001894766580000201

在本实施例中，因为在步骤14中已经得到了头颈部标准向量VNH＝(0.0,5.0,0.0)，在上一步骤中也得到了当前T时刻头颈部实时向量VNHS＝(-5.0,5.0,0.0)，则可以计算出头颈部夹角θ2T＝30.0°。In this embodiment, because the head and neck standard vector VNH =(0.0,5.0,0.0) has been obtained in step 14, the head and neck real-time vector VNHS =( -5.0, 5.0, 0.0), then the included head and neck angle θ2T =30.0° can be calculated.

步骤243、判断当前T时刻头颈部夹角θ2T是否大于头颈部夹角阈值，若当前T时刻头颈部夹角θ2T大于头颈部夹角阈值，则说明驾驶员当前头部有不正常晃动；否则，驾驶员当前头部未出现不正常晃动。Step 243: Determine whether the head and neck angle θ2T is greater than the head and neck angle threshold at the current T time. If the head and neck angle θ2T is greater than the head and neck angle threshold at the current T time, it means that the driver's current head and neck angle is greater than the head and neck angle threshold. There is abnormal shaking of the driver's head; otherwise, there is no abnormal shaking of the driver's head.

在本实施例中，头颈部夹角阈值为20°，在上一步中判断出驾驶员的当前T时刻头颈部晃动夹角为30°，那么可以判断此时驾驶员当前头部有不正常晃动。In this embodiment, the head and neck included angle threshold is 20°. In the previous step, it is judged that the driver's current T moment head and neck shaking included angle is 30°. Shake normally.

实施例二Embodiment two

对本发明提供的驾驶员危险驾驶行为检测方法进行试验验证，试验对象为公交车驾驶员，具体包括：The driver's dangerous driving behavior detection method provided by the present invention is tested and verified, and the test object is a bus driver, specifically including:

步骤I、部署实验场景，将Kinect设备安装到公交车驾驶室中；Step 1, deployment experiment scene, Kinect equipment is installed in the bus cab;

步骤II、驾驶员姿态校准，让驾驶员做出标准的驾驶员姿势并保持3秒，此时系统记录下标准姿势下的驾驶员骨骼姿态，并将此姿态保持入结构体BodyPoints中。Step II, driver posture calibration, let the driver take a standard driver posture and hold it for 3 seconds, at this time, the system records the driver's skeleton posture under the standard posture, and keeps this posture in the structure BodyPoints.

步骤III、检测驾驶员是否有危险驾驶行为，包括：Step III, detecting whether the driver has dangerous driving behavior, including:

(1)检测驾驶员手部是否离开方向盘(1) Detect whether the driver's hand leaves the steering wheel

通过不断增加驾驶员手离开方向盘的次数与频度测试本发明方法是否能实时准确检测。Whether the method of the present invention can be detected accurately in real time is tested by continuously increasing the number and frequency of the driver's hands leaving the steering wheel.

表1驾驶员手离开方向盘测试Table 1 Test with driver's hands off the steering wheel

频度(5分钟)Frequency (5 minutes)次数frequency识别次数Recognition times识别率Recognition rate252550505050100％100%505050505050100％100%10010050505050100％100%3003005050484896％96%6006005050464692％92%

表1中频度表示驾驶员手离开方向盘的频度，以5分钟为基准时间手离开方向盘的频度逐渐上升，当频度为25时，表示驾驶员每12秒手离开一次方向盘，当频度为600时，表示驾驶员每0.5秒手离开一次方向盘。次数表示在当前频度下手离开方向盘的测试次数。识别次数表示本方法对每次手离开方向盘的所有测试次数中，正确识别的次数，识别率为正确率。The frequency in Table 1 indicates the frequency at which the driver leaves the steering wheel. Taking 5 minutes as the base time, the frequency of hands leaving the steering wheel gradually increases. When the frequency is 25, it means that the driver leaves the steering wheel every 12 seconds. When the degree is 600, it means that the driver leaves the steering wheel every 0.5 seconds. The number of times represents the number of tests with hands off the steering wheel at the current frequency. The number of recognition times indicates the number of times of correct recognition among all the test times of each hand leaving the steering wheel in this method, and the recognition rate is the correct rate.

从表1中可以看出，当手离开方向盘的频度达到300次，即1秒驾驶员就把手离开方向盘时，本方法仍然可以达到96％的识别率；当手离开方向盘的频度达到600次，即0.5秒驾驶员就把手离开方向盘时，本发明提供的方法可以达到92％的识别率。As can be seen from Table 1, when the frequency of hands leaving the steering wheel reaches 300 times, that is, when the driver leaves the steering wheel in 1 second, this method can still achieve a recognition rate of 96%; when the frequency of hands leaving the steering wheel reaches 600 The first time, that is, when the driver leaves the steering wheel in 0.5 seconds, the method provided by the invention can reach a recognition rate of 92%.

(2)检测驾驶员是否有急转弯危险驾驶行为(2) Detect whether the driver has a dangerous driving behavior in a sharp turn

由于驾驶员急转弯是非常危险的行为，因此不在车辆行驶时进行测试，而让驾驶员在方向盘上做出急转弯的动作。Since it is very dangerous for the driver to make a sharp turn, the test is not performed while the vehicle is running, but the driver is asked to make a sharp turn on the steering wheel.

急转弯测试结果见表2，在这个实验中，仍然使用频度的增加作为急转动作的增长单位，急转动作的频度由5分钟内发生20次增长为5分钟600次。在频度为20时，驾驶员分别做出左急转和右急转动作20次，本发明提供的方法均成功识别出了这些急转弯动作。当频度上升为600时，驾驶员分别作出左急转和右急转动作40次，本发明提供的方法分别正确识别出了37次和38次，识别率达到了93％。The results of the sharp turn test are shown in Table 2. In this experiment, the increase in frequency is still used as the growth unit of the sharp turn action, and the frequency of the sharp turn action increases from 20 times in 5 minutes to 600 times in 5 minutes. When the frequency is 20, the driver makes sharp left turns and sharp right turns 20 times respectively, and the method provided by the present invention has successfully identified these sharp turns. When the frequency increased to 600, the driver made sharp left turns and sharp right turns 40 times respectively, and the method provided by the invention correctly recognized 37 times and 38 times respectively, and the recognition rate reached 93%.

表2急转弯测试结果Table 2 Sharp turn test results

Figure BDA0001894766580000221
Figure BDA0001894766580000221

(3)检测驾驶员是否有不正确坐姿(3) Detect whether the driver has an incorrect sitting posture

在这个测试中，驾驶员左右转动身体、左右晃动身体，使用本发明提供的方法检测驾驶员是否有不正确坐姿。In this test, the driver turns the body left and right, shakes the body left and right, and uses the method provided by the invention to detect whether the driver has an incorrect sitting posture.

测试结果见表3，实验频度仍然从5分钟20次上升为5分钟600次。当频度为20时，总体识别率可以达到100％，当平度上升为100时，有少数动作未被识别出来，当频度为600时，总体识别率仍然可以达到96％。The test results are shown in Table 3, and the experiment frequency still increased from 20 times in 5 minutes to 600 times in 5 minutes. When the frequency is 20, the overall recognition rate can reach 100%. When the flatness rises to 100, a few actions are not recognized. When the frequency is 600, the overall recognition rate can still reach 96%.

表3驾驶员不正确坐姿检测结果Table 3 Detection results of incorrect sitting posture of the driver

Figure BDA0001894766580000231
Figure BDA0001894766580000231

(4)检测驾驶员是否未专心驾驶(4) Detect whether the driver is not concentrating on driving

在这个测试中，申请人让驾驶员前后、左右晃动头部以模拟驾驶员在和周围人说话或其他未专心驾驶的行为。In this test, the applicant asks the driver to shake his head back and forth, side to side to simulate the driver talking to people around him or other inattentive driving behavior.

测试结果见表4，实验频度仍然从5分钟20次上升为5分钟600次。当频度为20时，总体识别率可以达到100％，当频度为100时，出现有少数动作未能被识别，当频度上升为600时，总体识别率仍然可以达到96％。The test results are shown in Table 4, and the experiment frequency still increased from 20 times in 5 minutes to 600 times in 5 minutes. When the frequency is 20, the overall recognition rate can reach 100%. When the frequency is 100, a few actions cannot be recognized. When the frequency increases to 600, the overall recognition rate can still reach 96%.

表4驾驶员未专心驾驶检测结果Table 4 The results of the driver's inattentive driving detection

Figure BDA0001894766580000232
Figure BDA0001894766580000232

本发明提供的方法可以有效并快速地检测和识别出常见的公交车危险驾驶行为，并且使用本方法进行前期准备的工作量非常少，可以快速开始对驾驶员的驾驶行为进行检测，且检测条件不受环境光线影响，很大程度上提升了基于视觉的驾驶员危险行为检测的可行性。The method provided by the present invention can effectively and quickly detect and identify common dangerous bus driving behaviors, and the workload of preliminary preparations using the method is very small, and the driver's driving behavior can be detected quickly, and the detection conditions Unaffected by ambient light, it greatly improves the feasibility of vision-based driver dangerous behavior detection.

Claims (4)

1. A method for detecting dangerous driving behavior of a driver, the method comprising:
step 1, when a driver holds two sides of a steering wheel by two hands to keep a standard posture, a Kinect camera is adopted to collect the skeleton posture of the driver at the moment, and a standard posture model is established;
the establishment of the standard attitude model comprises the following steps:
step 11, establishing a steering wheel standard envelope space shown as a formula I, wherein the unit of the coordinates of the steering wheel standard including the space on an X axis, a Y axis and a Z axis is unit length:
Figure FDA0001894766570000011
wherein (X)lefthand ,Ylefthand ,Zlefthand ) For the left-hand bone three-dimensional coordinates of the driver while maintaining the standard posture, (X)righthand ,Yrighthand ,Zrighthand ) For the three-dimensional coordinates of the right-hand bone of the driver when maintaining the standard attitude, SWxmin Is the coordinate minimum value, SW, in the X-axis direction of the standard envelope space of the steering wheelxmax Is the maximum value of the coordinate in the X-axis direction of the standard envelope space of the steering wheel, SWymin Is the minimum value of the coordinate in the Y-axis direction of the standard envelope space of the steering wheel, SWymax Is the maximum value of the coordinate in the Y-axis direction of the standard envelope space of the steering wheel, SWzmin Is the coordinate minimum value, SW, in the Z-axis direction of the standard envelope space of the steering wheelzmax Is the maximum value of the coordinate in the Z-axis direction of the standard envelope space of the steering wheel, Lfxp The projection distance H of the length of the steering wheel in the X-axis direction of the standard envelope space of the steering wheelfxp Is the projected distance, W, of the height of the steering wheel in the Y-axis direction in the standard envelope space of the steering wheelfxp The projection distance of the width of the steering wheel in the Z-axis direction in the standard envelope space of the steering wheel is taken as the distance;
step 12, calculating the standard steering frequency f of the steering wheel by adopting the formula Vfxp ：
Figure FDA0001894766570000021
Wherein R isfxp Is the projection distance of the radius of the steering wheel in the X-axis direction in the standard envelope space of the steering wheel,
Figure FDA0001894766570000022
unit is unit length, SRO For sharp steering detection sensitivity, SRO >0,Tfxp The time required for the steering wheel to normally turn for a half turn is s;
step 13, calculating a standard vector V of the shoulder of the driver by adopting the formula VIJB ：
Figure FDA0001894766570000023
Wherein (X)leftshoulder ,Yleftshoulder ,Zleftshoulder ) For the three-dimensional coordinates of the left shoulder joint of the driver while maintaining the standard attitude, (X)rightshoulder ,Yrightshoulder ,Zrightshoulder ) Three-dimensional coordinates of a right shoulder joint of the driver when the standard posture is maintained;
step 14, calculating the head and neck standard vector V of the driver by adopting the formula VIINH ：
VNH ＝(Xhead -Xneck ,Yhead -Yneck ,Zhead -Zneck ) Formula VII
Wherein (X)head ,Yhead ,Zhead ) For the three-dimensional head coordinates of the driver while maintaining the standard attitude, (X)neck ,Yneck ,Zneck ) Three-dimensional coordinates of the neck of the driver when the standard posture is kept;
step 2, repeatedly executing the step in the driving process of the driver, and detecting whether the driver has dangerous driving behaviors in the driving process:
utilize the Kinect camera to gather driver's skeleton gesture, with standard gesture model compare, detect whether driver has dangerous driving behavior, include:
step 21, judging whether the hand of the driver leaves the steering wheel or not according to whether the three-dimensional coordinates of the left bone and the three-dimensional coordinates of the right bone of the driver are both in the envelope space, if so, not leaving the steering wheel; otherwise, the hands of the driver leave the steering wheel;
step 22, judging whether the driver has dangerous driving behavior of sharp turning according to whether the current steering wheel rotating frequency is higher than the standard steering wheel rotating frequency, and if the current steering wheel rotating frequency is higher than the standard steering wheel rotating frequency, the driver has dangerous driving behavior of sharp turning; otherwise, the driver has no dangerous driving behavior of sharp turning;
step 23, judging whether the driver has an incorrect sitting posture at present according to an included angle between the current shoulder real-time vector of the driver and the shoulder standard vector at the present moment, if the included angle between the current shoulder real-time vector and the shoulder standard vector at the present moment is larger than a shoulder included angle threshold value, the current sitting posture of the driver is incorrect, otherwise, the current sitting posture of the driver is correct;
step 24, judging whether the head of the driver shakes abnormally according to the included angle between the real-time head and neck vector of the driver at the current moment and the standard head and neck vector, and if the included angle between the real-time head and neck vector of the driver at the current moment and the standard head and neck vector is larger than a neck threshold value, judging that the head of the driver shakes abnormally; otherwise, the head of the driver does not shake abnormally currently.
2. The method as claimed in claim 1, wherein the steering wheel rotation frequency f at the current time T in step 22 is determined by the driver's dangerous driving behavior detection methodTfxp Obtained using formula X:
Figure FDA0001894766570000041
wherein, tT Time of the current time T, TT-1 Is the time at the time T-1,
Figure FDA0001894766570000042
is the hand bone vector at the moment T-1,
Figure FDA0001894766570000043
Figure FDA0001894766570000044
is the axial coordinate of the left-hand bone at the T-1 moment X,
Figure FDA0001894766570000045
is the axial coordinate of the right hand bone at the T-1 moment X;
Figure FDA0001894766570000046
is the hand bone vector at the current time T,
Figure FDA0001894766570000047
Figure FDA0001894766570000048
is the X-axis coordinate of the left-hand bone at the T moment,
Figure FDA0001894766570000049
is the X-axis coordinate of the right-hand bone at T moment.
3. The method as claimed in claim 1, wherein in step 23, the included angle θ between the real-time shoulder vector of the driver at the current time T and the standard shoulder vector is determined1T Obtained using formula XII:
Figure FDA00018947665700000410
wherein, VJS Is a shoulder real-time vector at the current T moment, VJS ＝(XTrightshoulder -XTleftshoulder ,YTrightshoulder -YTleftshoulder ,ZTrightshoulder -ZTleftshoulder )，(XTleftshoulder ,YTleftshoulder ,ZTleftshoulder ) Is the three-dimensional coordinate of the left shoulder joint at the current T moment, (X)Trightshoulder ,YTrightshoulder ,ZTrightshoulder ) The three-dimensional coordinates of the right shoulder joint at the current time T.
4. The method as claimed in claim 1, wherein said step 24 is performed by using an angle θ between the real-time head and neck vector of the driver at the current T time and the standard head and neck vector2T Obtained using formula XIV:
Figure FDA00018947665700000411
wherein, VNHS Is a head and neck real-time vector at the current T moment, VNHS ＝(XThead -XTneck ,YThead -YTneck ,ZThead -ZTneck )，(XThead ,YThead ,ZThead ) As the three-dimensional coordinates of the head bone at the current time T, (X)Tneck ,YTneck ,ZTneck ) The three-dimensional coordinates of the neck bone at the current T moment.
CN201811487007.5A2018-12-062018-12-06 A method for detecting driver's dangerous driving behaviorExpired - Fee RelatedCN109543651B (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN201811487007.5ACN109543651B (en)2018-12-062018-12-06 A method for detecting driver's dangerous driving behavior

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN201811487007.5ACN109543651B (en)2018-12-062018-12-06 A method for detecting driver's dangerous driving behavior

Publications (2)

Publication NumberPublication Date
CN109543651A CN109543651A (en)2019-03-29
CN109543651Btrue CN109543651B (en)2022-11-22

Family

ID=65852982

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN201811487007.5AExpired - Fee RelatedCN109543651B (en)2018-12-062018-12-06 A method for detecting driver's dangerous driving behavior

Country Status (1)

CountryLink
CN (1)CN109543651B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN111368696A (en)*2020-02-282020-07-03淮阴工学院 Detection method and system for illegal driving behavior of hazardous chemicals transport vehicle based on visual collaboration
CN111449652B (en)*2020-05-062022-11-29北方工业大学 A construction safety monitoring method and device based on brain wave analysis
CN112287795B (en)*2020-10-222023-09-01北京百度网讯科技有限公司Abnormal driving gesture detection method, device, equipment, vehicle and medium
CN112339764A (en)*2020-11-042021-02-09杨华勇New energy automobile driving posture analysis system based on big data
CN114715159B (en)*2022-04-142024-09-17重庆理工大学Steering wheel hand-off detection method and system based on artificial intelligence learning
CN115471826B (en)*2022-08-232024-03-26中国航空油料集团有限公司Method and device for judging safe driving behavior of aviation fueller and safe operation and maintenance system
CN116965781B (en)*2023-04-282024-01-05南京晓庄学院 A method and system for monitoring driver's vital signs and driving behavior

Citations (4)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN105551182A (en)*2015-11-262016-05-04吉林大学Driving state monitoring system based on Kinect human body posture recognition
JP2016200910A (en)*2015-04-082016-12-01日野自動車株式会社Driver state determination device
CN108446678A (en)*2018-05-072018-08-24同济大学A kind of dangerous driving behavior recognition methods based on skeleton character
US10081366B1 (en)*2015-05-042018-09-25Carnegie Mellon UniversitySensor-based assessment of attention interruptibility

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
JP2016200910A (en)*2015-04-082016-12-01日野自動車株式会社Driver state determination device
US10081366B1 (en)*2015-05-042018-09-25Carnegie Mellon UniversitySensor-based assessment of attention interruptibility
CN105551182A (en)*2015-11-262016-05-04吉林大学Driving state monitoring system based on Kinect human body posture recognition
CN108446678A (en)*2018-05-072018-08-24同济大学A kind of dangerous driving behavior recognition methods based on skeleton character

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于驾驶行为的汽车主动安全技术研究;肖献强等;《中国机械工程》;20101010(第19期);全文*
车辆典型危险行驶状态识别与检测研究进展;刘通等;《中国安全科学学报》;20171015(第10期);全文*

Also Published As

Publication numberPublication date
CN109543651A (en)2019-03-29

Similar Documents

PublicationPublication DateTitle
CN109543651B (en) A method for detecting driver&#39;s dangerous driving behavior
CN104200192B (en)Driver&#39;s gaze detection system
CN114742863B (en) Method and device with slip detection and correction function
US10655978B2 (en)Controlling an autonomous vehicle based on passenger behavior
WO2022095440A1 (en)Self-driving-oriented human-machine collaborative perception method and system
CN109271914A (en)Detect method, apparatus, storage medium and the terminal device of sight drop point
García et al.Driver monitoring based on low-cost 3-D sensors
Selim et al.AutoPOSE: Large-scale Automotive Driver Head Pose and Gaze Dataset with Deep Head Orientation Baseline.
CN104570731A (en)Uncalibrated human-computer interaction control system and method based on Kinect
Borghi et al.Embedded recurrent network for head pose estimation in car
CN112489425A (en)Vehicle anti-collision early warning method and device, vehicle-mounted terminal equipment and storage medium
CN110069979A (en)Controller&#39;s condition detection method, apparatus and system
DE102020213102A1 (en) EQUIPMENT AND SYSTEM FOR AN INTELLIGENT HELMET
CN109906069A (en)A kind of intelligent wheelchair system with medical monitoring and response function
Long et al.A high speed eye tracking system with robust pupil center estimation algorithm
CN109522887A (en)A kind of Eye-controlling focus method, apparatus, equipment and storage medium
CN112789020A (en)Visualization method and system for intelligent wheelchair
Shen et al.CoCAtt: A cognitive-conditioned driver attention dataset
WO2020228224A1 (en)Face part distance measurement method and apparatus, and vehicle-mounted terminal
CN108256487B (en)Driving state detection device and method based on reverse dual-purpose
Guasconi et al.A low-cost implementation of an eye tracking system for driver's gaze analysis
WO2023103145A1 (en)Head pose truth value acquisition method, apparatus and device, and storage medium
Pasarica et al.Remote control of a robotic platform based on hand gesture recognition
JP3686418B2 (en) Measuring device and method
CN117734707A (en) Driver fatigue status detection system and method

Legal Events

DateCodeTitleDescription
PB01Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination
GR01Patent grant
GR01Patent grant
CF01Termination of patent right due to non-payment of annual fee

Granted publication date:20221122

CF01Termination of patent right due to non-payment of annual fee
[8]ページ先頭
©2009-2025 Movatter.jp