Movatterモバイル変換


[0]ホーム

URL:


CN116185198A - Cabin interaction method, cabin interaction equipment, vehicle and storage medium - Google Patents

Cabin interaction method, cabin interaction equipment, vehicle and storage medium
Download PDF

Info

Publication number
CN116185198A
CN116185198ACN202310187409.8ACN202310187409ACN116185198ACN 116185198 ACN116185198 ACN 116185198ACN 202310187409 ACN202310187409 ACN 202310187409ACN 116185198 ACN116185198 ACN 116185198A
Authority
CN
China
Prior art keywords
driver
cabin
module
behavior
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310187409.8A
Other languages
Chinese (zh)
Inventor
王岳超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Changan Automobile Co Ltd
Original Assignee
Chongqing Changan Automobile Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Changan Automobile Co LtdfiledCriticalChongqing Changan Automobile Co Ltd
Priority to CN202310187409.8ApriorityCriticalpatent/CN116185198A/en
Publication of CN116185198ApublicationCriticalpatent/CN116185198A/en
Pendinglegal-statusCriticalCurrent

Links

Images

Classifications

Landscapes

Abstract

The invention belongs to the technical field of man-machine interaction of automobile cabins, and particularly relates to a cabin interaction method, equipment, a vehicle and a storage medium. A cabin interaction method, comprising: the perception module collects behavior information of a driver and uploads the behavior information of the driver to the prediction module; according to a preset behavior prediction model, a prediction module predicts and analyzes the behavior of the driver to obtain the probability P of the driver operating the central control screen; when the probability P exceeds a preset execution threshold T, outputting an operation request M to an execution module; and according to the operation request M, the execution module executes corresponding operation. The operation of the driver is predicted according to the behavior of the driver, and the execution module controls the automatic ejection of the control button on the central control screen of the cabin, so that the driver can more conveniently perform key operation, and the driving safety is improved.

Description

Cabin interaction method, cabin interaction equipment, vehicle and storage medium
Technical Field
The invention belongs to the technical field of man-machine interaction of automobile cabins, and particularly relates to a cabin interaction method, equipment, a vehicle and a storage medium.
Background
Intelligent cabins are an important direction of intelligent development of automobiles, and along with the trend of the hardware configuration and performance of the automobiles, the intellectualization of cabins is an important component for measuring an automobile. The intelligent cabin, particularly the central control large screen, bears most of human-computer interface functions at present and bears the core functions of vehicle-to-vehicle interaction. Touch of the vehicle function buttons is focused on the vehicle display screen. The key position is difficult to confirm during driving due to the influence of factors such as key position change caused by illumination reflection and updating iteration of a vehicle system, and the safety problem is caused by the fact that the driver's sight is separated from the road surface due to the complicated operation process of searching for related keys.
Disclosure of Invention
The purpose of the invention is that: the method, the device, the vehicle and the storage medium for cabin interaction aim to provide a cabin interaction method, device, vehicle and storage medium, operation of a driver is predicted according to the behavior of the driver, and an execution module controls a control button to be automatically popped up on a central control screen of a cabin, so that the driver can more conveniently perform key operation, and driving safety is improved.
In order to achieve the technical purpose, the invention adopts the following technical scheme:
in a first aspect, an embodiment of the present application provides a cabin interaction method, applied to a cabin interaction device, where the device includes a sensing module, a prediction module, and an execution module, where the prediction module is configured to predict an operation of a driver according to a driver behavior, and the method includes:
the perception module acquires behavior information of a driver and uploads the behavior information of the driver to the prediction module;
according to a preset behavior prediction model, the prediction module performs prediction analysis on the driver behavior to obtain the probability P of the driver operating the central control screen;
when the probability P exceeds a preset execution threshold T, outputting an operation request M to the execution module;
and according to the operation request M, the execution module executes corresponding operation.
With reference to the first aspect, in some optional embodiments, the sensing module collects behavior information of the driver, including:
the sensing module comprises a DMS camera, a CAN bus and an in-vehicle environment sensor; the DMS camera is used for collecting operation images and audio of a driver, the CAN bus is used for collecting vehicle running environment information, and the in-vehicle sensor is used for collecting in-vehicle environment information.
With reference to the first aspect, in some optional embodiments, according to a preset behavior prediction model, the prediction module performs prediction analysis on the driver behavior to obtain a probability P of the driver operating the central control screen, where the method includes:
and constructing the behavior prediction model, namely taking a display screen image of a driver operation cabin as a positive sample, taking a display screen of a driver driving and a display screen of a cabin which is not operated in the vehicle as a negative sample, performing migration learning training by using MobileNet, classifying the types of results output by the behavior prediction model into two types, and only calculating 0-1 probability distribution with and without operation will to obtain the probability P of the driver operation center control screen.
With reference to the first aspect, in some optional embodiments, according to a preset behavior prediction model, the prediction module performs prediction analysis on the behavior of the driver to obtain a probability P that the driver operates the central control screen, where the method includes:
the input information of the behavior prediction model data is from the information of the CAN bus and a preset cabin operation record, past vehicle state information and cabin operation history record are used as first input information, current vehicle CAN bus state information and cabin current state information are used as second input information, the first input information and the second input information are structured and then are converted by adopting a matrix similarity algorithm, and the input information of the behavior prediction model is generated.
With reference to the first aspect, in some optional implementations, when the probability P exceeds a preset execution threshold T, outputting an operation request M to the execution module includes:
when the probability P exceeds a preset execution threshold T, the prediction module obtains a vehicle running state according to the vehicle running environment information and the vehicle internal environment information, and determines an operation request M to be executed according to a preset driver history operation record.
With reference to the first aspect, in some optional embodiments, according to the operation request M, the executing module performs a corresponding operation, including:
and according to the operation request M, the execution module calls a preset operation key list, and recommended operation keys are popped up on a central control screen of the cabin.
In a second aspect, embodiments of the present application further provide a cabin interaction device, which is characterized in that: the cabin interaction device comprises a perception module, a prediction module, an execution module and a storage module, wherein the prediction module is used for predicting the operation of a driver according to the behavior of the driver, a computer program is stored in the storage module, and when the computer program is executed by the prediction module and the execution module, the cabin interaction device is enabled to execute the method.
In a third aspect, an embodiment of the present application further provides a vehicle, where the vehicle includes a vehicle body and the cabin interaction device described above, and the cabin interaction device is disposed on the vehicle.
In a fourth aspect, embodiments of the present application further provide a computer readable storage medium, where a computer program is stored, which when run on a computer, causes the computer to perform the above-mentioned method.
The invention adopting the technical scheme has the following advantages:
the sensing module collects behavior information of a driver in the cabin and running state information of the vehicle, the prediction module predicts operation of the driver, and when the probability P that the driver can perform central control screen operation exceeds a preset operation execution threshold T, the execution module automatically retrieves a corresponding key list and displays the key list on a central control screen. The driver can conveniently and rapidly finish the key operation on the central control screen of the cabin, and the situation that the sight of the driver leaves the road for a long time due to the fact that the driver searches for the operation key on the central control screen is avoided, so that the driving safety is improved.
Drawings
The invention can be further illustrated by means of non-limiting examples given in the accompanying drawings;
fig. 1 is a block diagram of a cabin interaction device provided in an embodiment of the present application;
fig. 2 is a schematic flow chart of a cabin interaction method provided in an embodiment of the present application;
fig. 3 is a schematic diagram of an operation key calling step of a cabin interaction method according to an embodiment of the present application;
the main reference numerals are as follows:
10. cabin interaction equipment; 11. a perception module; 12. a prediction module; 13. and executing the module.
Detailed Description
The present invention will be described in detail below with reference to the drawings and the specific embodiments, wherein like or similar parts are designated by the same reference numerals throughout the drawings or the description, and implementations not shown or described in the drawings are in a form well known to those of ordinary skill in the art. In addition, directional terms such as "upper", "lower", "top", "bottom", "left", "right", "front", "rear", etc. in the embodiments are merely directions with reference to the drawings, and are not intended to limit the scope of the present invention.
As shown in fig. 1, an embodiment of the present application provides acabin interaction device 10, and thecabin interaction device 10 may include aperception module 11, aprediction module 12, anexecution module 13, and a storage module, where theprediction module 12 is configured to predict an operation of a driver according to a driver behavior.
In this embodiment, thesensing module 11 collects behavior information of a driver in the cabin and running state information of the vehicle, theprediction module 12 predicts the operation of the driver, and when the probability P that the driver will perform the operation of the central control screen exceeds the preset operation execution threshold T, theexecution module 13 automatically retrieves the corresponding key list and displays the key list on the central control screen. The driver can conveniently and rapidly finish the key operation on the central control screen of the cabin, and the situation that the sight of the driver leaves the road for a long time due to the fact that the driver searches for the operation key on the central control screen is avoided, so that the driving safety is improved.
It will be appreciated that theperception module 11 may include an audio video acquisition sensor, a CAN bus, and in-vehicle environment sensors, which in turn may include
The storage module stores a computer program which, when executed by theprediction module 12 and theexecution module 13, causes thecabin interaction device 10 to perform the method described below.
As shown in fig. 2, an embodiment of the present application further provides a cabin interaction method, which may include the following steps:
step 110: thesensing module 11 collects behavior information of a driver and uploads the behavior information of the driver to theprediction module 12;
step 120: according to a preset behavior prediction model, theprediction module 12 performs prediction analysis on the behavior of the driver to obtain the probability P of the driver operating the central control screen;
step 130: when the probability P exceeds a preset execution threshold T, outputting an operation request M to theexecution module 13;
step 140: according to the operation request M, theexecution module 13 executes a corresponding operation.
In this embodiment, thesensing module 11 senses and collects the behavior information of the driver in the cabin, uploads the behavior information of the driver to the behavior prediction model of theprediction module 12, performs prediction analysis on the behavior of the driver, determines the operation wish of the driver, obtains the probability P of the driver operating the central control screen, outputs the operation request M to theexecution module 13 when the probability P exceeds the preset execution threshold T, and theexecution module 13 invokes the corresponding operation key list and displays the operation key list on the central control screen, so that the driver can press the operation button when lifting his hand.
As an alternative embodiment, thesensing module 11 collects behavior information of the driver, including:
thesensing module 11 comprises a DMS camera, a CAN bus and an in-vehicle environment sensor; the DMS camera is used for collecting operation images and audio of a driver, the CAN bus is used for collecting vehicle running environment information, and the in-vehicle sensor is used for collecting in-vehicle environment information.
In this embodiment, the DMS camera collects historical operation images and audio of the driver, and uses the collected historical operation images and audio as sample data for creating a behavior prediction model, and inputs the behavior prediction model to predict the intention of the driver to control the central control screen according to operation images and audio information of the driver during the driving process collected by the DMS camera.
The CAN bus acquires vehicle running environment information, including vehicle speed information, geographical position information and vehicle window state information.
The in-vehicle environment sensor may include a cabin temperature sensor, an ambient temperature sensor, an air conditioning status sensor, and other status data sensors.
As an optional implementation manner, according to a preset behavior prediction model, the prediction module performs prediction analysis on the behavior of the driver to obtain a probability P that the driver operates the central control screen, where the method includes:
the input information of the behavior prediction model data is from the information of the CAN bus and a preset cabin operation record, past vehicle state information and cabin operation history record are used as first input information, current vehicle CAN bus state information and cabin current state information are used as second input information, the first input information and the second input information are structured and then are converted by adopting a matrix similarity algorithm, and the input information of the behavior prediction model is generated.
As shown in fig. 3, in this embodiment, the behavior prediction model is constructed including two paths of input information, and the first portion is: and combining CAN bus information and cabin operation records, and taking past vehicle state information and cabin operation history records as one path of input of a behavior prediction model. And taking the current vehicle CAN bus state information and the cabin current state information as the other path of input. The two parts of data can form two state tables after being structured, the similarity between the current state and the state of the vehicle and the cabin before the operation of the key can be calculated by adopting a matrix similarity algorithm, and a similarity vector taking the key of the cabin as an item is generated. The similarity vector is sequenced from large to small, and then the first three items are taken as input information of the behavior prediction model, and theexecution module 13 prepares a corresponding key display UI in the background.
As an alternative embodiment, according to a preset behavior prediction model, theprediction module 12 performs a prediction analysis on the driver behavior to obtain a probability P of the driver operating the central control screen, including:
and constructing the behavior prediction model, namely taking a display screen image of a driver operation cabin as a positive sample, taking a display screen of a driver driving and a display screen of a cabin which is not operated in the vehicle as a negative sample, performing migration learning training by using MobileNet, classifying the types of results output by the behavior prediction model into two types, and only calculating 0-1 probability distribution with and without operation will to obtain the probability P of the driver operation center control screen.
It can be understood that, the MobileNet is an image classification probability calculation method model, and the MobileNet can output results under multiple classifications and modify the output quantity according to requirements. In other embodiments of the present solution, only the probability of the situation with the willingness to operate may be calculated, and the types of results output by the behavior prediction model are only one type.
In this embodiment, after the prediction model is constructed, the operation intention of the driver is predicted according to the operation action of the driver, and as the operation of the driver proceeds, the probability P of the driver operating the center control screen gradually increases when there is an operation intention.
As an alternative embodiment, when the probability P exceeds a preset execution threshold T, outputting an operation request M to theexecution module 13 includes:
when the probability P exceeds a preset execution threshold T, theprediction module 12 obtains a vehicle running state according to the vehicle running environment information and the in-vehicle environment information, and determines an operation request M to be executed in combination with a preset driver history operation record.
In the present embodiment, when the predicted probability P of the driver operating the center screen exceeds the preset execution threshold T, it is possible to identify that the driver has a desire to operate the vehicle center screen, and theprediction module 12 determines that the driver wants to execute the operation request M based on the vehicle running state and the driver history.
As an alternative embodiment, according to the operation request M, theexecution module 13 performs corresponding operations, including:
according to the operation request M, theexecution module 13 invokes a preset operation key list, and pops up recommended operation keys on the central control screen of the cabin.
In this embodiment, theprediction module 12 sends the predicted operation request M to theexecution module 13, and theexecution module 13 invokes the corresponding operation key list and pops up and displays the recommended operation key on the central control screen of the cabin.
To facilitate understanding of the present solution, the driver is presented with a description of the adjustment of the temperature in the vehicle. Theperception module 11 collects the driver behavior information, and theprediction module 12 predicts that the driver has the operation willingness to adjust the cabin temperature by combining the historical behavior information of the driver when the perception module monitors that the attention of the driver is transferred to the central control screen area and the driver has the operation central control willingness by the aid of the audio-video file. The in-car environment sensor of thesensing module 11 acquires state data such as the current cabin temperature T-cabin, the ambient temperature sensor T-env, the vehicle Speed, the air conditioning state AC-on/off, the vehicle window state Win-on/off and the like, and forms a state matrix A_cur and B_val respectively when the state data such as the cabin temperature sensor value T-cabin, the ambient temperature sensor value T-env, the vehicle Speed, the air conditioning state AC-on/off, the vehicle window state Win-on/off and the like are combined with a driver history operation button. And calculating the similarity S of the current state and the past positive sample state by adopting the pearson correlation coefficient, and then taking three positive sample key operation results t1, t2 and t3 with the highest similarity by utilizing the sequence of the similarity from large to small.
When the cabin display module receives the positive sample M output by the driver behavior prediction system, the buttons t1, t2 and t3 are sequentially displayed on the left side of the display screen according to recommended buttons t1, t2 and t3 output by the button recommendation system.
Wherein, t1=air conditioning temperature setting key, t2=air conditioning air output volume key, t3=window lifting key. And popping up the side, close to the driver, of the central control screen according to the sequence from top to bottom, so that a rapid operation function is realized.
In one embodiment of the invention, the volume adjustment assembly is invoked into theexecution module 13 in response to thesensing module 11 detecting that there is a person talking in the vehicle and the current cabin entertainment system output sound intensity exceeds a calibrated limit. When thesensing module 11 detects that the attention of the driver is transferred to the central control screen area and the driver has a intention to operate the central control, the volume adjusting main switch pops up at the side of the central control screen, close to the driver, so that the function of quickly operating and adjusting the volume in the cabin is realized.
In this embodiment, the memory module may be, but is not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc. In this embodiment, the storage module may be configured to store collected behavior audio, video files, vehicle driving environment information, in-vehicle environment information, a preset behavior prediction model, and the like of the driver. Of course, the storage module may also be used to store a program, and theexecution module 13 executes the program after receiving the execution instruction.
It will be appreciated that thecabin interaction device 10 structure shown in fig. 1 is only a schematic structural diagram, and that thecabin interaction device 10 may also include more components than those shown in fig. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.
It should be noted that, for convenience and brevity of description, the specific working process of thecabin interaction device 10 described above may refer to the corresponding process of each step in the foregoing method, and will not be described in detail herein.
The embodiment of the application also provides a vehicle. The vehicle comprises a vehicle body and thecabin interaction device 10 as described in the above embodiments. Thecabin interaction device 10 is deployed on a vehicle body. Thecabin interaction device 10 can be used for realizing the cabin interaction method, and can improve the convenience of a control screen of a driver in the driving process, thereby being beneficial to improving driving safety.
Embodiments of the present application also provide a computer-readable storage medium. The computer readable storage medium has stored therein a computer program which, when run on a computer, causes the computer to perform the cabin interaction method as described in the above embodiments.
From the foregoing description of the embodiments, it will be apparent to those skilled in the art that the present application may be implemented in hardware, or by means of software plus a necessary general hardware platform, and based on this understanding, the technical solution of the present application may be embodied in the form of a software product, where the software product may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disc, a mobile hard disk, etc.), and includes several instructions to cause a computer device (may be a personal computer, an interaction device, or a network device, etc.) to perform the methods described in the respective implementation scenarios of the present application.
In summary, the embodiments of the present application provide a cabin interaction method, a cabin interaction device, a vehicle and a storage medium. In this scheme, thesensing module 11 collects behavior information of a driver in the cabin and running state information of the vehicle, theprediction module 12 predicts operation of the driver, and when the probability P that the driver will perform the operation of the central control screen exceeds a preset operation execution threshold T, theexecution module 13 automatically retrieves a corresponding key list and displays the key list on the central control screen. The driver can conveniently and rapidly finish the key operation on the central control screen of the cabin, and the situation that the sight of the driver leaves the road for a long time due to the fact that the driver searches for the operation key on the central control screen is avoided, so that the driving safety is improved.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus, system, and method may be implemented in other manners as well. The above-described apparatus, systems, and method embodiments are merely illustrative, for example, flow charts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. In addition, the functional modules in the embodiments of the present application may be integrated together to form a single part, or each module may exist alone, or two or more modules may be integrated to form a single part.
The foregoing is merely exemplary embodiments of the present application and is not intended to limit the scope of the present application, and various modifications and variations may be suggested to one skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims (9)

CN202310187409.8A2023-03-022023-03-02Cabin interaction method, cabin interaction equipment, vehicle and storage mediumPendingCN116185198A (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN202310187409.8ACN116185198A (en)2023-03-022023-03-02Cabin interaction method, cabin interaction equipment, vehicle and storage medium

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN202310187409.8ACN116185198A (en)2023-03-022023-03-02Cabin interaction method, cabin interaction equipment, vehicle and storage medium

Publications (1)

Publication NumberPublication Date
CN116185198Atrue CN116185198A (en)2023-05-30

Family

ID=86442134

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN202310187409.8APendingCN116185198A (en)2023-03-022023-03-02Cabin interaction method, cabin interaction equipment, vehicle and storage medium

Country Status (1)

CountryLink
CN (1)CN116185198A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN119938766A (en)*2025-04-082025-05-06吉林大学 A structured data generation method, device, terminal device and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN119938766A (en)*2025-04-082025-05-06吉林大学 A structured data generation method, device, terminal device and storage medium
CN119938766B (en)*2025-04-082025-07-22吉林大学 A structured data generation method, device, terminal device and storage medium

Similar Documents

PublicationPublication DateTitle
US9501693B2 (en)Real-time multiclass driver action recognition using random forests
CN107709127B (en) Driving assistance method and device, automatic driving control device, vehicle, recording medium
EP3272611B1 (en)Information processing system, information processing method, and program
KR20170128397A (en) System and method for adjusting user interface for user attention and driving status
WO2019011268A1 (en)Game theory-based driver auxiliary system decision-making method and system, and the like
CN114684152B (en) Driving experience data processing method, device, vehicle and medium
CN112905004B (en)Gesture control method and device for vehicle-mounted display screen and storage medium
US20200114925A1 (en)Interaction device, interaction method, and program
EP4506852A1 (en)Car computer avatar interaction method and system, and vehicle and storage medium
CN112947740A (en)Human-computer interaction method based on motion analysis and vehicle-mounted device
CN116185198A (en)Cabin interaction method, cabin interaction equipment, vehicle and storage medium
CN115268334A (en)Vehicle window control method, device, equipment and storage medium
US11427216B2 (en)User activity-based customization of vehicle prompts
CN113760423A (en)Vehicle with intelligent user interface
KR20200102307A (en)Method for recognizing emotion of driver and apparatus for the same
CN114815676A (en)Generation and presentation of interpretations relating to behavior of an automation control system
CN117112633A (en)Active interaction method, system and storage medium based on intelligent cabin
CN112109645A (en)Method and system for providing assistance to a vehicle user
CN112959945A (en)Vehicle window control method and device, vehicle and storage medium
CN113501004B (en) Gesture-based control method, device, electronic device and storage medium
CN119058502A (en) Vehicle control method, device, equipment and storage medium
JP7524775B2 (en) Vehicle allocation support system and vehicle allocation support method
Akita et al.Analysis and synthesis of driving behavior based on mode segmentation
KR102286569B1 (en)Smart car see-through display control system and method thereof
US20230316826A1 (en)Information processing apparatus, computer-readable storage medium, and information processing method

Legal Events

DateCodeTitleDescription
PB01Publication
PB01Publication
SE01Entry into force of request for substantive examination
SE01Entry into force of request for substantive examination

[8]ページ先頭

©2009-2025 Movatter.jp