Disclosure of Invention
The invention aims to solve the problems in the prior art, creatively discloses an intelligent management auxiliary system and method, and effectively solves the problems that traffic accidents caused by fatigue driving cannot be effectively prevented, accurate driving advice and health reminding cannot be provided, and more comfortable travel experience cannot be provided due to the prior art, and meanwhile, the risks of data leakage and abuse are avoided.
The embodiment of the invention discloses an intelligent management auxiliary system, which comprises: the vehicle-mounted device comprises a pre-driving auxiliary detection module, a data acquisition module, a physiological and psychological analysis module and an evaluation result module, wherein the cloud server comprises a voice program and a database;
The auxiliary detection module before driving is used for playing a voice command according to the voice program, a driver performs auxiliary inspection on the vehicle according to the voice command to obtain safety data of the vehicle, the safety data are uploaded to the database, and the vehicle state is judged according to the safety data;
The data acquisition module is used for acquiring a plurality of video data, a plurality of voice data and a plurality of driving data in the vehicle for a plurality of times and uploading the video data, the voice data and the driving data to the database;
The physiological and psychological analysis module is used for carrying out physiological and psychological analysis on the plurality of video data for a plurality of times through an artificial intelligence technology to obtain a plurality of physiological data and a plurality of psychological data of a driver, and uploading the physiological data and the psychological data to the database;
And the evaluation result module is used for judging whether the driver is suitable for continuing driving for a plurality of times according to the voice data, the driving data, the physiological data and the psychological data, obtaining a plurality of evaluation results, uploading the evaluation results to the database, and informing related personnel and/or suspending the driving permission of the driver when the evaluation result is that the driver is unsuitable for continuing driving.
Optionally, the data acquisition module includes: the system comprises a camera for acquiring the video data, a microphone for acquiring the voice data and a positioning system for acquiring the driving data.
Optionally, the vehicle-mounted device further comprises a face recognition module, a safety question-answering module, a subjective safety evaluation module and a privacy protection module, and the cloud server further comprises a face database, a preset question library and a data analysis module, wherein the face database comprises image data corresponding to a plurality of drivers respectively;
The face recognition module is used for carrying out framing processing on the plurality of video data to obtain a plurality of target image data, comparing the plurality of target image data with the image data respectively corresponding to the plurality of drivers to obtain a comparison result, uploading the comparison result to the database, and judging whether the drivers have the driving permission according to the comparison result;
The safety question-answering module is used for playing a plurality of preset questions in the preset question library, a driver answers the plurality of preset questions to obtain a plurality of answer voice data, the plurality of answer voice data are converted into a plurality of answer text data by using a voice recognition technology and uploaded into the database, and the data analysis module analyzes the answer text data by using the artificial intelligence technology; wherein, the obtaining a plurality of answer voice data specifically includes: extracting the plurality of answer speech data from the plurality of speech data; the plurality of voice data in the evaluation result module are specifically the plurality of answer voice data;
The subjective safety evaluation module is used for carrying out safety evaluation on the safety of a driver according to the answer text data to obtain an evaluation result, uploading the evaluation result to the database, and determining whether the driver is allowed to drive according to the evaluation result;
The privacy protection module is used for ensuring the safety of data in the data acquisition module, the safety question answering module, the subjective safety evaluation module, the physiological and psychological analysis module and the evaluation result module.
Optionally, the cloud server further includes the safety questioning and answering module, a remote monitoring and intervention module and a driving behavior analysis module;
the remote monitoring and intervention module performs real-time comparison in the command large screen according to the data in the database, so as to realize remote monitoring; judging whether the driving process of the driver is safe or not according to the data in the database, reminding the driver if the driving process of the driver is unsafe, and/or carrying out active tracking by contacting with nearby administrative staff, and remotely controlling the vehicle to further realize remote intervention;
And the driving behavior analysis module analyzes the driving behavior of the driver for a plurality of times according to the data in the database, obtains a plurality of analysis results, uploads the analysis results to the database, and judges whether the driving behavior of the driver is compliant according to the analysis results.
The embodiment of the invention also discloses an intelligent management auxiliary method which is applied to the vehicle-mounted equipment, wherein the vehicle-mounted equipment is communicated with a cloud server through a wireless communication technology, the cloud server comprises a voice program and a database, and the method comprises the following steps:
Playing a voice command according to the voice program, and carrying out auxiliary inspection on the vehicle by a driver according to the voice command to obtain safety data of the vehicle, uploading the safety data to the database, and judging the state of the vehicle according to the safety data;
collecting a plurality of video data, a plurality of voice data and a plurality of driving data in the vehicle for a plurality of times, and uploading the video data, the voice data and the driving data to the database;
performing physiological and psychological analysis on the video data for a plurality of times through an artificial intelligence technology to obtain physiological data and psychological data of a driver, and uploading the physiological data and psychological data to the database;
judging whether the driver is suitable for continuing driving for a plurality of times according to the voice data, the driving data, the physiological data and the psychological data, obtaining a plurality of evaluation results, uploading the evaluation results to the database, and informing relevant personnel and/or suspending driving permission of the driver when the evaluation results are that the driver is not suitable for continuing driving.
Optionally, the collecting multiple video data, multiple voice data and multiple driving data in the vehicle for multiple times and uploading to the database includes:
The video data are collected for a plurality of times through a camera and uploaded to the database;
collecting the voice data for a plurality of times through a microphone and uploading the voice data to the database;
and acquiring the driving data for a plurality of times through a positioning system and uploading the driving data to the database.
Optionally, the cloud server further comprises a face database, a preset problem library and a data analysis module, wherein the face database comprises image data corresponding to a plurality of drivers respectively;
the steps of collecting a plurality of video data, a plurality of voice data and a plurality of driving data in the vehicle for a plurality of times, uploading the video data, the voice data and the driving data to the database comprise:
The video data are subjected to framing processing to obtain a plurality of target image data, the target image data are respectively compared with the image data corresponding to the drivers respectively to obtain a comparison result, the comparison result is uploaded to the database, and whether the drivers have the driving permission is judged according to the comparison result;
Playing a plurality of preset questions in the preset question library, enabling a driver to answer the plurality of preset questions to obtain a plurality of answer voice data, converting the plurality of answer voice data into a plurality of answer text data by using a voice recognition technology, uploading the answer text data into the database, and analyzing the answer text data by the data analysis module through the artificial intelligence technology; wherein, the obtaining a plurality of answer voice data specifically includes: extracting the plurality of answer speech data from the plurality of speech data;
The multiple video data are subjected to multiple times of physiological and psychological analysis through an artificial intelligence technology to obtain multiple physiological data and multiple psychological data of a driver, and the multiple physiological data and the multiple psychological data are uploaded to the database, and then the multiple physiological and psychological data are included in the database, wherein the multiple physiological and psychological data comprise: and carrying out safety evaluation on the safety of the driver according to the answer text data to obtain an evaluation result, uploading the evaluation result to the database, and determining whether to allow the driver to drive according to the evaluation result.
The embodiment of the invention also discloses an intelligent management auxiliary method which is applied to a cloud server, wherein the cloud server comprises a database, and the method comprises the following steps:
According to the data in the database, real-time comparison is carried out in the command large screen, so that remote monitoring is realized; judging whether the driving process of the driver is safe or not according to the data in the database, reminding the driver if the driving process of the driver is unsafe, and/or actively tracking by contacting with nearby administrative staff, and remotely controlling the vehicle to further realize remote intervention;
And analyzing the driving behaviors of the driver for a plurality of times according to the data in the database, obtaining a plurality of analysis results, uploading the analysis results to the database, and judging whether the driving behaviors of the driver are compliant according to the analysis results.
The embodiment of the invention also discloses electronic equipment, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
The memory is used for storing a computer program;
The processor is configured to implement the method according to the embodiment of the present invention when executing the program stored in the memory.
The embodiment of the invention also discloses a computer program product which is stored in a storage medium and is executed by at least one processor to realize the method according to the embodiment of the invention.
Embodiments of the present invention also disclose a computer-readable storage medium having instructions stored thereon, which when executed by one or more processors, cause the processors to perform the method according to the embodiments of the present invention.
The embodiment of the invention has the following advantages:
In the embodiment of the invention, the physiological and psychological states of the driver can be comprehensively monitored through the auxiliary detection module before driving, the data acquisition module, the physiological and psychological analysis module and the evaluation result module, the driving safety can be improved on the premise that the problem that the traffic accident caused by fatigue driving cannot be effectively prevented is solved, the health of the driver is ensured, and the operation cost can be reduced under the condition that the fine and accident compensation caused by violation, fatigue driving and other reasons are reduced.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
Referring to fig. 1, a schematic structural diagram of an intelligent management assistance system provided in an embodiment of the present invention is shown, including: the vehicle-mounted device comprises a pre-driving auxiliary detection module, a data acquisition module, a physiological and psychological analysis module and an evaluation result module, wherein the cloud server comprises a voice program and a database;
In the embodiment of the invention, the vehicle-mounted equipment comprises an embedded system (embedded equipment) which is used for running the checking process, collecting video, voice and driving data, carrying out interactive question answering of safety problems and the like. The in-vehicle devices may include embedded processors, cameras, microphones, positioning systems, and the like. The embedded processor is used for running the checking process, collecting video, voice and driving data, carrying out interactive question answering and the like of the safety problems; the camera and the microphone are used for collecting video and voice data of a driver, and the positioning system is used for collecting driving data of a vehicle. The embedded system has the computing power meeting the actual requirement, and is provided with a physiological and psychological detection module based on an AI (ARTIFICIAL INTELLIGENCE ) technology, namely the vehicle-mounted equipment has the AI capability, and can calculate physiological and psychological indexes based on uninterrupted video and give out proper conclusion.
The cloud server is used for storing data, running a data analysis module and providing functions of a face database, a voice program, a preset question library, safety question interactive questions and answers and the like. The cloud server can comprise a database, a data analysis module, a face database, a voice program, a preset question library and a safety question-answering module. The database is used for storing data acquired in the vehicle, data processed according to an AI technology, evaluation results and the like; the data analysis module is used for carrying out statistics and analysis on the data stored in the database so as to provide more accurate safety evaluation and suggestion for drivers; image data in the face library can be managed and updated to ensure the accuracy and timeliness of information in the face library; the safety question-answering module is used for receiving the answer of the driver, understanding the answer of the driver through natural language processing technology and giving out the next question. Because the storage space of the database is limited, the long-term availability of the data in the database can be ensured by the technologies such as tape backup and the like, and the database can be cleared regularly according to the service requirement.
By adopting the wireless communication technology, data transmission and control between the vehicle-mounted equipment and the cloud server can be realized.
The auxiliary detection module before driving is used for playing a voice command according to the voice program, a driver performs auxiliary inspection on the vehicle according to the voice command to obtain safety data of the vehicle, the safety data are uploaded to the database, and the vehicle state is judged according to the safety data;
The driver and the vehicle are safe at the same time, so that the driver and the vehicle are in an optimal state at the same time, the driver can be helped to check the vehicle before driving the vehicle through the auxiliary detection module before driving, the vehicle is ensured to be checked in place, and potential safety hazards can be found timely. Therefore, before the driver drives the vehicle, the voice command needs to be played according to the voice program in the cloud server, so that the driver can carry out necessary inspection on the vehicle according to the voice command to judge the state of the vehicle, and meanwhile, the requirement on collecting safety data information of the vehicle can be met.
Specifically, the main implementation mode of the auxiliary detection module before driving is to play a voice command to guide a driver to check the vehicle, the driver and the robot can carry out one-to-one check and confirm processes under the assistance of the voice command, important check steps are not needed to be skipped, all safety data obtained according to checking are uploaded to a database so as to be used for the vehicle system to judge the state of the vehicle in advance, wherein the auxiliary detection module before driving can be combined with a remote control system of the vehicle to ensure that the important check steps are not skipped, authorization can be carried out after all check steps are completed, the driver can drive the vehicle, and in the process, answers given by the driver are needed to be judged through an AI technology to judge whether the answers meet requirements or not.
The data acquisition module is used for acquiring a plurality of video data, a plurality of voice data and a plurality of driving data in the vehicle for a plurality of times and uploading the video data, the voice data and the driving data to the database;
In the embodiment of the invention, in order to ensure the safety of a driver in the driving process, a data acquisition module is used for continuously acquiring a plurality of video data, a plurality of voice data and a plurality of driving data in the vehicle in real time in the driving process. The video data includes behavior data of the driver, face data of the driver, and the like, for example, the behavior data of the driver can refer to illegal operations such as smoking, holding a steering wheel by one hand, answering a call, and the like of the driver in the video data, and certainly can refer to normal operations of the driver, and whether the operation of the driver is illegal is obtained by processing and analyzing the behavior data through an AI technology; the voice data comprise answer voice data for a driver to answer a preset question in a safety question answering module; the driving data may refer to data such as vehicle speed, vehicle specific position, etc. And uploading various data acquired in real time to a database of a cloud server for subsequent calling.
The physiological and psychological analysis module is used for carrying out physiological and psychological analysis on the plurality of video data for a plurality of times through an artificial intelligence technology to obtain a plurality of physiological data and a plurality of psychological data of a driver, and uploading the physiological data and the psychological data to the database;
In the embodiment of the invention, the artificial intelligence technology used for carrying out physiological and psychological analysis on the video data can be a computer vision and machine learning technology, specifically, the physiological analysis on the video data can be carried out, the physiological index of a driver can be monitored by utilizing the computer vision technology, the facial features of the driver are identified by using a camera and an image processing algorithm, and the data of the physiological index are extracted from the facial features, so that the physiological data are obtained; the video is subjected to psychological analysis, the behavior and expression of the driver can be analyzed by using machine learning technology, different facial expressions and actions and voice and language characteristics related to different emotion states are identified by training models, so that psychological states of the driver are deduced, and psychological data can be obtained and can be used for evaluating the emotion states, the attention level and the behavior characteristics of the driver.
Specifically, physiological data detected based on face video data are mainly composed of four items including blood pressure (including high pressure and low pressure), blood oxygen saturation, heart rate and respiratory rate; psychological data detected based on face video data currently has 16 main dimensions, including 12 emotion types: aggressiveness, stress (anxiety), suspicion, balance, confidence, vitality, self-regulation, inhibition, neurology, subsidence, happiness, and 4 psychophysiological parameters: camber, stability, satisfaction, brain activity cycle. Based on video signals of human faces, vestibules and the like, the video micro-vibration signals can be detected, which represent psychological activities of people, and different dimensional psychological activity values can be obtained through different vibration frequencies and amplitudes.
The physiological and psychological analysis module can reduce the operation cost, and can reduce fine and accident compensation caused by reasons such as violations, fatigue driving and the like through real-time monitoring and analysis of the driver behaviors, so that the operation cost is reduced, for example, the conditions such as unfocused attention, fatigue driving and the like can be found through physiological and psychological detection analysis based on face video data, and violations caused by psychological state fluctuation such as red light running, lane line driving and the like can be indirectly and effectively avoided; the physiological and psychological analysis module can improve driving safety, for example, collected video data can be compared with individual or collective average indexes of drivers, skin color change can be detected, the video data can comprise whole facial tags, such as frequent yawning, long-time eye closing and the like, and the video data can be used as auxiliary judgment, so that drunk driving judgment, especially drunk driving type, can be realized; the physiological and psychological analysis module can ensure the health of a driver, and can remind the driver to seek medical attention in time when a problem occurs through real-time monitoring of physiological data of the driver, so that the risk of illness is reduced; the physiological and psychological analysis module can improve the service quality, monitor the physiological and psychological states of the driver in real time, timely prevent dangerous occurrence, minimize the running risk, meet the requirements of a cargo owner on cargo transportation, and ensure the physical and psychological health of the driver, namely improve the service quality of cargo transportation.
In the embodiment of the invention, the physiological and psychological detection analysis is based on the face video data, physiological data and psychological data can be obtained through the processing and analysis of the video data acquired by the camera, and the psychological state of the driver can be comprehensively predicted by combining the behavioral data of the driver in the video data; in the prior art, physiological data of a driver is often acquired through a contact sensor, and the data acquired by a camera are only used for indexes such as facial expression, pupil size, head posture, blink times, eyeball position, eyeball motion track, eyeball motion speed, eyeball residence time and the like.
The physiological data and the psychological data are uploaded to the database, and specifically, the physiological database and the psychological database are contained in the database and are used for storing the physiological data and the psychological data respectively, so that the physiological data and the psychological data can be conveniently acquired from the corresponding databases during subsequent calling.
And the evaluation result module is used for judging whether the driver is suitable for continuing driving for a plurality of times according to the voice data, the driving data, the physiological data and the psychological data, obtaining a plurality of evaluation results, uploading the evaluation results to the database, and informing related personnel and/or suspending the driving permission of the driver when the evaluation result is that the driver is unsuitable for continuing driving.
In the embodiment of the invention, in order to ensure that attention and treatment can be timely obtained and the driving safety is ensured when the state of the driver is poor, in the driving process of the driver, the evaluation result module is used for judging whether the driver is suitable for continuing driving, and when the evaluation result shows that the driver is not suitable for continuing driving, actions are immediately taken, for example, related personnel are notified through a mobile phone APP or a telephone dialing mode, driving permission of the driver is suspended, and the like, so that the driving safety is ensured. Here, the relevant person may be a leader, an administrator, a monitoring person, or the like, and it should be noted that the embodiment of the present invention is not limited to the setting of the relevant person.
In the existing post detection, besides the existing behavior and action detection and physiological analysis, the mental state analysis is added, and the important point is more accurate mental state analysis and combination of the mental state analysis and the physiological state analysis, so that joint analysis is performed to give more accurate judgment.
In the embodiment of the invention, the physiological and psychological states of the driver can be comprehensively monitored through the auxiliary detection module before driving, the data acquisition module, the physiological and psychological analysis module and the evaluation result module, the driving safety can be improved on the premise that the problem that the traffic accident caused by fatigue driving cannot be effectively prevented is solved, the health of the driver is ensured, and the operation cost can be reduced under the condition that the fine and accident compensation caused by violation, fatigue driving and other reasons are reduced.
Referring to fig. 2, an overall structure schematic diagram of an intelligent management assistance system provided in an embodiment of the present invention is shown, a vehicle-mounted device further includes a security question-answering module, a subjective security evaluation module, a privacy protection module, and a cloud server further includes a driving behavior analysis module and a remote monitoring and intervention module.
In one embodiment of the present invention, the data acquisition module includes: the system comprises a camera for acquiring the video data, a microphone for acquiring the voice data and a positioning system for acquiring the driving data.
Specifically, the data acquisition module can always acquire data in real time before a driver drives or in the driving process, and can be used for real-time analysis of real-time data by other subsequent modules through real-time collection of the driver data, so that targeted management advice can be provided for transportation enterprises, the operation efficiency and market competitiveness of the enterprises are improved, innovation and application of related technologies such as artificial intelligence, big data and the like can be promoted, and more values are brought to society.
In the prior art, in order to directly collect physiological data of a driver, a contact type sensor is generally used for data collection, for example, the contact type physiological sensor can be a blood pressure meter, an oxygen indicator and the like, and in the embodiment of the invention, the data collection module does not need to use the contact type physiological sensor to collect the data.
The camera in the embodiment of the invention may include an infrared camera, an infrared thermal imaging camera, and the like, and the positioning system may be a beidou system.
In one embodiment of the invention, the vehicle-mounted device further comprises a face recognition module, a safety question-answering module, a subjective safety evaluation module and a privacy protection module, and the cloud server further comprises a face database, a preset question library and a data analysis module, wherein the face database comprises image data corresponding to a plurality of drivers respectively;
The face recognition module is used for carrying out framing processing on the plurality of video data to obtain a plurality of target image data, comparing the plurality of target image data with the image data respectively corresponding to the plurality of drivers to obtain a comparison result, uploading the comparison result to the database, and judging whether the drivers have the driving permission according to the comparison result;
The safety question-answering module is used for playing a plurality of preset questions in the preset question library, a driver answers the plurality of preset questions to obtain a plurality of answer voice data, the plurality of answer voice data are converted into a plurality of answer text data by using a voice recognition technology and uploaded into the database, and the data analysis module analyzes the answer text data by using the artificial intelligence technology; wherein, the obtaining a plurality of answer voice data specifically includes: extracting the plurality of answer speech data from the plurality of speech data; the plurality of voice data in the evaluation result module are specifically the plurality of answer voice data;
The subjective safety evaluation module is used for carrying out safety evaluation on the safety of a driver according to the answer text data to obtain an evaluation result, uploading the evaluation result to the database, and determining whether the driver is allowed to drive according to the evaluation result;
The privacy protection module is used for ensuring the safety of data in the data acquisition module, the safety question answering module, the subjective safety evaluation module, the physiological and psychological analysis module and the evaluation result module.
Specifically, in the face recognition module, 10 frames of photos containing the face of the driver, namely a plurality of target image data, can be obtained by carrying out frame processing on the video data, the 10 frames of photos are respectively compared with the image data in the face database to obtain a comparison result, and whether the driver has driving permission for the vehicle or not can be judged according to the comparison result, so that the safety of the vehicle is further ensured. It should be noted that, the embodiment of the present invention is not limited to the setting of the number of the target image data.
In the safety question-answering module, the answer of the driver can be converted into a text through a voice recognition technology, namely, answer voice data is converted into answer text data, then the answer text data is sent to a database, the answer text data is analyzed through a data analysis module, a plurality of questions can be automatically broadcasted in the module, the driver needs to answer the broadcasted questions, the answer of the driver does not have an answer defined by a standard, the answer of the driver is processed and analyzed through an AI technology, the next broadcasted question is judged according to the result obtained through the processing and analysis, and the AI technology used in the safety question-answering module can be specifically natural language processing technology. Specifically, according to the answer of the driver, what is the next question is determined, a background maintains a set of scenes, and the question-answer routes generated by different answers are different. The preset questions in the preset question library may include driver identity, trip information, whether there is a violation (e.g., drinking) and the like. In the safe interactive question answering process, the judgment basis for automatically broadcasting the next question can be the following three types: 1) No speech was made for 20 seconds; 2) After speaking is detected, silence is detected within 5 seconds; 3) After speaking is detected, the next is automatically broadcasted more than 5 seconds. It should be noted that, the embodiment of the present invention is not limited to the setting of the preset problem and the determination of broadcasting the next problem.
In addition, answer voice data are not only required to be uploaded to a database in the data acquisition module, but also required to be uploaded to the local module, and are used for designing the questions in the safety question-answering module and storing the questions in a preset question library as backups, so that the designed questions can be more fit and practical, and the overall accuracy and practicability of the intelligent management auxiliary system can be improved.
In the subjective safety evaluation module, the driver is evaluated according to the answer of the driver to judge whether the driver is suitable for driving or not so as to perform safety evaluation, the driver is not required to be evaluated through other data such as physiological data, psychological data and the like, the safety of the driver is only required to be evaluated according to subjective answer text data so as to obtain an evaluation result, and whether the driver is allowed to drive or not is determined according to the evaluation result.
On the premise of ensuring driving safety, in order to ensure privacy data protection in the use process of technologies such as face recognition, voice recognition and the like, privacy benefits of drivers are fully considered, a privacy protection module adopts an encryption technology to protect the safety of the data in the process of collecting and processing the data of the drivers, only authorized personnel can access the data, and the privacy protection module cannot involve decryption process, but in a cloud server, the data are summarized and the data decryption process is involved in later invoking. The module can effectively protect privacy rights and interests of drivers and avoid data leakage and abuse.
In one embodiment of the present invention, the cloud server further includes the safety questioning and answering module, a remote monitoring and intervention module, and a driving behavior analysis module;
the remote monitoring and intervention module performs real-time comparison in the command large screen according to the data in the database, so as to realize remote monitoring; judging whether the driving process of the driver is safe or not according to the data in the database, reminding the driver if the driving process of the driver is unsafe, and/or carrying out active tracking by contacting with nearby administrative staff, and remotely controlling the vehicle to further realize remote intervention;
And the driving behavior analysis module analyzes the driving behavior of the driver for a plurality of times according to the data in the database, obtains a plurality of analysis results, uploads the analysis results to the database, and judges whether the driving behavior of the driver is compliant according to the analysis results.
In one embodiment, the security question-answering module is not only in the vehicle-mounted device, but also in the cloud server, so that the security interactive question-answering can be completed in an off-line state, and the preset question library is updated in an on-line state, so that the security of data can be improved.
In order to improve driving efficiency and safety, in the remote monitoring and intervention module, real-time monitoring and analysis are performed on video data, wherein the video data comprise behavior data of a driver, and the behavior of the driver can be monitored and analyzed in real time, so that more reasonable driving advice is provided for the driver, and driving efficiency and safety are improved. Specifically, the driver in the normal state will not make driving advice, and different driving advice is given for detecting the occurrence of a problem, for example, driving advice in three cases of continuing driving but needing to pay attention to observation (mild), stopping and resting in the next service area (moderate), stopping nearby and getting contact with the outside (severe) can be classified to remind the driver. In the process of reminding the driver, the embodiment of the invention is more dependent on the AI technology, and is obviously different from the prior art which mainly depends on the acquisition equipment, and mainly comprises the following three points: 1) The AI technology can monitor and analyze a large amount of video data in real time, can more accurately identify the behavior state of a driver and timely give corresponding driving advice, and in contrast, the behavior of the driver cannot be accurately analyzed and judged only by the acquisition equipment; 2) The AI technology can provide more personalized and highly adaptive driving advice according to different driving scenes and driver behaviors, and the AI system can continuously learn and optimize and gradually improve the recognition and analysis capability of the driver behaviors, so that more accurate driving advice is provided for a driver; 3) The automatic monitoring and analysis of the driver behavior can be realized by relying on the AI technology, manual intervention is not needed, the AI system can automatically give corresponding driving advice according to preset rules and models, and the intelligent degree of the system is improved.
The remote intervention is realized in the remote monitoring and intervention module, the speed limiting module for actively tracking and remotely controlling the vehicle is contacted with the nearby road staff to achieve the purpose of intervention, and the personnel authority level for realizing the remote intervention is relatively higher in order to ensure the safety of drivers and vehicles in the driving process.
The driving behavior analysis module is mainly used for sorting and summarizing a large amount of original data and is used for later training. Specifically, through opening the data of the driving behavior analysis module and the vehicle-mounted equipment, the driving situation is recorded in the driving process, and whether the current behavior of the driver is compliant is fully analyzed. Meanwhile, through big data collection and arrangement, the reverse case and the optimal case which are most prone to generating potential safety hazards are formed and used for carrying out safety evaluation and safety education related programs.
In the embodiment of the invention, the latest AI technology such as physiological and psychological analysis, face recognition, voice recognition, behavior recognition and the like is adopted to realize the comprehensive monitoring of the driver. Meanwhile, a certain mechanism is adopted for checking the safety of the vehicle. In the network layer of the application, a mode of combining a cloud server and vehicle-mounted equipment is adopted, so that real-time transmission and remote control of data are realized. In addition, privacy protection of a driver is fully considered while driving safety is guaranteed.
Referring to fig. 3, a step flowchart of an intelligent management assistance method provided in an embodiment of the present invention is applied to a vehicle-mounted device, where the vehicle-mounted device communicates with a cloud server through a wireless communication technology, the cloud server includes a voice program and a database, and the method includes:
Step 301, playing a voice command according to the voice program, and carrying out auxiliary inspection on a vehicle by a driver according to the voice command to obtain safety data of the vehicle, uploading the safety data to the database, and judging the state of the vehicle according to the safety data;
step 302, collecting a plurality of video data, a plurality of voice data and a plurality of driving data in the vehicle for a plurality of times, and uploading the video data, the voice data and the driving data to the database;
step 303, performing physiological and psychological analysis on the video data for a plurality of times through an artificial intelligence technology to obtain physiological data and psychological data of a driver, and uploading the physiological data and psychological data to the database;
And 304, judging whether the driver is suitable for continuing driving for a plurality of times according to the voice data, the driving data, the physiological data and the psychological data, obtaining a plurality of evaluation results, uploading the evaluation results to the database, and informing related personnel and/or suspending the driving permission of the driver when the evaluation results are that the driver is not suitable for continuing driving.
In the embodiment of the invention, the following beneficial effects can be realized through the steps: 1) And the driving safety is improved. Through the real-time monitoring and early warning of the psychological and physiological states of the driver, traffic accidents caused by fatigue driving can be effectively prevented, and therefore driving safety is improved. 2) The health of the driver is ensured. Through the real-time monitoring to driver's physiological index, can in time remind the driver to seek medical advice when the problem appears to reduce the sick risk. 3) And the service quality is improved. Through the real-time monitoring of the physiological and psychological states of the driver, the requirements of the cargo owner on cargo transportation can be met, and therefore the service quality is improved. 4) And the operation cost is reduced. By means of real-time monitoring and analysis of the physiological and psychological states of the driver, fine and accident compensation caused by violations, fatigue driving and other reasons can be reduced, and therefore operation cost is reduced.
In one embodiment of the present invention, the step 302 of collecting a plurality of video data, a plurality of voice data and a plurality of driving data in the vehicle a plurality of times and uploading the video data, the voice data and the driving data to the database includes:
The video data are collected for a plurality of times through a camera and uploaded to the database;
collecting the voice data for a plurality of times through a microphone and uploading the voice data to the database;
and acquiring the driving data for a plurality of times through a positioning system and uploading the driving data to the database.
In an embodiment of the invention, the cloud server further comprises a face database, a preset problem database and a data analysis module, wherein the face database contains image data corresponding to a plurality of drivers respectively;
Specifically, whether before a driver drives or during the driving process, data can be always collected in real time, and through the real-time collection of the driver data, the method can be used for carrying out subsequent real-time analysis on the data, and further can provide targeted management advice for transportation enterprises, improve the operation efficiency and market competitiveness of the enterprises, and can also promote innovation and application of artificial intelligence, big data and other technologies, thereby bringing more value to society.
In the prior art, in order to directly collect physiological data of a driver, a contact type sensor is generally used for data collection, for example, the contact type physiological sensor can be a blood pressure meter, an oxygen indicator and the like, and in the embodiment of the invention, the data collection module does not need to use the contact type physiological sensor to collect the data. The camera in the embodiment of the invention may include an infrared camera, an infrared thermal imaging camera, and the like, and the positioning system may be a beidou system.
Step 302, after collecting a plurality of video data, a plurality of voice data and a plurality of driving data in the vehicle for a plurality of times and uploading the video data, the voice data and the driving data to the database, includes:
The video data are subjected to framing processing to obtain a plurality of target image data, the target image data are respectively compared with the image data corresponding to the drivers respectively to obtain a comparison result, the comparison result is uploaded to the database, and whether the drivers have the driving permission is judged according to the comparison result;
Playing a plurality of preset questions in the preset question library, enabling a driver to answer the plurality of preset questions to obtain a plurality of answer voice data, converting the plurality of answer voice data into a plurality of answer text data by using a voice recognition technology, uploading the answer text data into the database, and analyzing the answer text data by the data analysis module through the artificial intelligence technology; wherein, the obtaining a plurality of answer voice data specifically includes: extracting the plurality of answer speech data from the plurality of speech data;
The multiple video data are subjected to multiple times of physiological and psychological analysis through an artificial intelligence technology to obtain multiple physiological data and multiple psychological data of a driver, and the multiple physiological data and the multiple psychological data are uploaded to the database, and then the multiple physiological and psychological data are included in the database, wherein the multiple physiological and psychological data comprise: and carrying out safety evaluation on the safety of the driver according to the answer text data to obtain an evaluation result, uploading the evaluation result to the database, and determining whether to allow the driver to drive according to the evaluation result.
The safety in the driving process can be further improved through AI technologies such as face recognition, voice recognition and the like.
Referring to fig. 4, a flowchart of steps of another intelligent management assistance method provided in an embodiment of the present invention is applied to a cloud server, where the cloud server includes a database, and the method includes:
Step 401, comparing in real time in a command large screen according to the data in the database, so as to realize remote monitoring; judging whether the driving process of the driver is safe or not according to the data in the database, reminding the driver if the driving process of the driver is unsafe, and/or actively tracking by contacting with nearby administrative staff, and remotely controlling the vehicle to further realize remote intervention;
And 402, analyzing the driving behavior of the driver for a plurality of times according to the data in the database, obtaining a plurality of analysis results, uploading the analysis results to the database, and judging whether the driving behavior of the driver is compliant according to the analysis results.
In summary, the embodiment of the invention adopts the latest AI technology, such as physiological and psychological analysis, face recognition, voice recognition, behavior recognition and the like, so as to realize comprehensive monitoring of the driver. Meanwhile, a certain mechanism is adopted for checking the safety of the vehicle. In the network layer of the application, a mode of combining a cloud server and vehicle-mounted equipment is adopted, so that real-time transmission and remote control of data are realized.
In order to provide a better understanding of the embodiments of the present invention to those skilled in the art, a specific example is described below. Referring to fig. 5, a flowchart illustrating overall steps of an intelligent management assistance method provided in an embodiment of the present invention is applied to a vehicle-mounted device, where the method specifically may include the following steps:
Step 501, starting the embedded device;
Step 502, broadcasting: welcome the use;
Step 503, broadcasting: please see the camera;
step 504, obtaining 10 frames of photos for face recognition;
step 505, call 1# interface to judge whether driver himself;
In this step, a result of face recognition can be obtained, and the result is uploaded to the cloud server no matter whether the face recognition passes or not.
Step 506, broadcasting a prompt;
if the driver is not himself, broadcasting: please recheck; if the driver himself, broadcasting: the verification of the inventor passes, and the safety evaluation link is entered below.
Step 507, entering a safety question answering, physiological and psychological state analysis link;
Step 5071, broadcasting a problem and starting recording; storing the safe question and answer result to the local;
and the question and answer result obtained by the sound recording is converted into text data by using an ASR (Automatic Speech Recognition, speech recognition) technology and is uploaded to a cloud server. And step 5071 needs to be repeated until step 508 is entered.
Step 5072, starting video recording; executing a python script to evaluate the physiological state, updating the physiological evaluation result and uploading the physiological evaluation result to a physiological database; and starting a psychological assessment program to evaluate the psychological state; monitoring abnormal psychological monitoring conditions through a No. 2 interface callback, updating psychological assessment results and uploading the psychological assessment results to a psychological database;
The above steps 5071 and 5072 are performed simultaneously.
Step 508, taking values from the physiological database and the psychological database, and judging whether the evaluation of the physiological and psychological states is completed;
step 509, if the evaluation of the physiological and psychological states is completed, whether the driver is allowed to drive is determined according to the evaluation; if not, returning to step 508;
step 5010, incomplete assessment of physiological+psychological state, judging whether the process exceeds 5 minutes; if the time exceeds 5 minutes, broadcasting: the state cannot be determined once;
step 5011, if not more than 5 minutes, broadcasting: the status of you is being evaluated, please return to step 508 later and once every 10 seconds, and re-judge;
Step 5012, if the driver is allowed to drive, continuing the physiological and psychological assessment, and judging whether the physiological and psychological states of the driver are abnormal every 20 seconds;
in the driving process, the step 5012 is continuously performed;
step 5013, if the physiological and psychological states of the driver are abnormal, broadcasting prompt voice and judging whether the abnormality is serious; if serious, calling a 3# interface, and notifying a leader;
Step 5014, reporting abnormal results of physiological and psychological monitoring to a cloud server, and returning to step 5012 to continuously analyze the physiological and psychological states of a driver in real time;
it should be noted that, the foregoing formulation of the interval time may be adjusted accordingly according to the actual situation, and the embodiment of the present invention is not limited.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
In addition, the embodiment of the invention also provides electronic equipment, which comprises: the processor, the memory, store the computer program on the memory and can run on the processor, this computer program realizes each process of the above-mentioned intelligent management auxiliary method embodiment when being carried out by the processor, and can reach the same technical result, in order to avoid repetition, will not be repeated here.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, realizes the processes of the above-mentioned intelligent management assistance method embodiment, and can achieve the same technical effects, and in order to avoid repetition, the description is omitted here. The computer readable storage medium is, for example, a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a magnetic disk or an optical disk.
The embodiment of the present invention further provides a computer program product, which is stored in a storage medium, and the program product is executed by at least one processor to implement the respective processes of the above-mentioned embodiment of the intelligent management assistance method, and the same technical effects can be achieved, so that repetition is avoided, and a detailed description is omitted herein.
Fig. 6 is a schematic diagram of a hardware structure of an electronic device implementing various embodiments of the present invention.
The electronic device 600 includes, but is not limited to: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, processor 610, and power supply 611. It will be appreciated by those skilled in the art that the electronic device structure shown in fig. 6 is not limiting of the electronic device and that the electronic device may include more or fewer components than shown, or may combine certain components, or a different arrangement of components. In the embodiment of the invention, the electronic equipment comprises, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer and the like.
It should be understood that, in the embodiment of the present invention, the radio frequency unit 601 may be used to receive and send information or signals during a call, specifically, receive downlink data from a base station, and then process the downlink data with the processor 610; and, the uplink data is transmitted to the base station. Typically, the radio frequency unit 601 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 601 may also communicate with networks and other devices through a wireless communication system.
The electronic device provides wireless broadband internet access to the user via the network module 602, such as helping the user to send and receive e-mail, browse web pages, and access streaming media, etc.
The audio output unit 603 may convert audio data received by the radio frequency unit 601 or the network module 602 or stored in the memory 609 into an audio signal and output as sound. Also, the audio output unit 603 may also provide audio output (e.g., a call signal reception sound, a message reception sound, etc.) related to a specific function performed by the electronic device 600. The audio output unit 603 includes a speaker, a buzzer, a receiver, and the like.
The input unit 604 is used for receiving audio or video signals. The input unit 604 may include a graphics processor (Graphics Processing Unit, GPU) 6041 and a microphone 6042, the graphics processor 6041 processing image data of still pictures or video obtained by an image capturing apparatus (such as a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 606. The image frames processed by the graphics processor 6041 may be stored in the memory 609 (or other storage medium) or transmitted via the radio frequency unit 601 or the network module 602. Microphone 6042 may receive sound and can process such sound into audio data. The processed audio data may be converted into a format output that can be transmitted to the mobile communication base station via the radio frequency unit 601 in the case of a telephone call mode.
The electronic device 600 also includes at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 6061 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 6061 and/or the backlight when the electronic device 600 moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for recognizing the gesture of the electronic equipment (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; the sensor 605 may also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which are not described herein.
The display unit 606 is used to display information input by a user or information provided to the user. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in a form of a Liquid crystal display (Liquid CRYSTAL DISPLAY, LCD), an Organic Light-Emitting Diode (OLED), or the like.
The user input unit 607 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the electronic device. Specifically, the user input unit 607 includes a touch panel 6071 and other input devices 6072. Touch panel 6071, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on touch panel 6071 or thereabout using any suitable object or accessory such as a finger, stylus, or the like). The touch panel 6071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 610, and receives and executes commands sent from the processor 610. In addition, the touch panel 6071 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The user input unit 607 may include other input devices 6072 in addition to the touch panel 6071. Specifically, other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a track ball, a mouse, and a joystick, which are not described herein.
Further, the touch panel 6071 may be overlaid on the display panel 6061, and when the touch panel 6071 detects a touch operation thereon or thereabout, the touch operation is transmitted to the processor 610 to determine a type of a touch event, and then the processor 610 provides a corresponding visual output on the display panel 6061 according to the type of the touch event. Although in fig. 6, the touch panel 6071 and the display panel 6061 are two independent components for implementing the input and output functions of the electronic device, in some embodiments, the touch panel 6071 and the display panel 6061 may be integrated to implement the input and output functions of the electronic device, which is not limited herein.
The interface unit 608 is an interface to which an external device is connected to the electronic apparatus 600. For example, the external devices may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 608 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic apparatus 600 or may be used to transmit data between the electronic apparatus 600 and an external device.
The memory 609 may be used to store software programs as well as various data. The memory 609 may mainly include a storage program area that may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and a storage data area; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, the memory 609 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
The processor 610 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 609, and calling data stored in the memory 609, thereby performing overall monitoring of the electronic device. The processor 610 may include one or more processing units; preferably, the processor 610 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 610.
The electronic device 600 may also include a power supply 611 (e.g., a battery) for powering the various components, and preferably the power supply 611 may be logically coupled to the processor 610 via a power management system that performs functions such as managing charging, discharging, and power consumption.
In addition, the electronic device 600 includes some functional modules, which are not shown, and will not be described herein.
In yet another embodiment of the present invention, as shown in fig. 7, a computer readable storage medium 701 is provided, where instructions are stored, when the computer readable storage medium runs on a computer, to cause the computer to perform the intelligent management assistance method described in the above embodiment.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.
The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.