Disclosure of Invention
Aiming at the situation, in order to overcome the defects of the prior art, the invention provides a method and a device for carrying out passenger clearance judgment based on subway carriage videos, which effectively solve the problems that a station operator is required to carry out carriage-by-carriage inspection in the process of passenger clearance treatment of subway carriages and carriage passenger clearance scene optimization calculation through video identification is lacked.
The invention comprises an interface input/output module, wherein the interface input/output module is connected with an image analysis and calculation module, and the image analysis and calculation module is connected with an integrated analysis module;
the interface input and output module, the image analysis and calculation module and the integrated analysis module are sequentially connected.
Preferably, an image preprocessing module is connected between the interface input/output module and the image analysis and calculation module.
Preferably, the image preprocessing module is connected with a deep learning analysis and calculation module, and the deep learning analysis and calculation module is connected with the integrated analysis module.
Preferably, the image analysis and calculation module and the deep learning and calculation module are both connected with an autonomous video acceleration card.
Preferably, the method for carrying out passenger clearing judgment based on subway carriage video comprises the following steps:
step S1: the method comprises the steps of obtaining video streams of subway carriage pictures by accessing video stream addresses of carriage cameras, and converting the video streams into video picture sequences;
step S2: performing image stabilization analysis on the acquired carriage image sequence;
step S3: performing optical compensation operation on the picture sequence processed in the step S2;
step S4: performing image background modeling on the picture sequence processed in the step S3;
Step S5: performing passenger detection on the picture sequence modeled in the step S4 and the image sequence processed in the step S3;
Step S6: carrying out passenger identification based on deep learning on the picture sequence acquired in the step S3;
step S7: carrying out integrated detection on the result of passenger detection in the step S5 and the result of passenger identification in the step S6;
step S8: dividing a carriage area of each carriage of the train;
step S9: detecting the position of a passenger in the carriage according to the region division result of the carriage in the step S8 and the result of the integrated learning in the step S7;
Step S10: obtaining the final number of passengers and the positions of the passengers relative to the carriage according to the carriage position detection result in the step S9;
Step S11: according to the detection result of the step S10, if no passenger exists in the carriage, the passenger clearing is judged to be successful;
step S12: and according to the detection result of the step S10, if the passenger exists in the carriage, judging that the passenger clearing is not finished.
Preferably, the step S6 specifically includes:
step S6-1: establishing a data model of completion of subway carriage passenger cleaning and incompletion of subway carriage passenger cleaning;
step S6-2: establishing an image recognition model of subway carriage passenger detection based on deep learning, and training a carriage passenger detection model by utilizing the data established in the step S6-1 to obtain parameters of the carriage passenger detection model;
Step S6-3: taking the image sequence obtained in the step S3 as an input parameter, and carrying out passenger detection by utilizing the carriage passenger detection model trained in the step S6-2 to obtain the number of passengers in the input image sequence and the coordinate value of the relative image of the passengers;
Step S6-4: and (3) according to the number of passengers and the coordinate value of the passenger relative image in the image sequence detected in the step (S6-3), if the image detection result has no passenger, the passenger identification result based on the deep learning is no passenger, and if the image detection result has a passenger, the passenger identification result based on the deep learning is a passenger, and the passenger identification result comprises the number of passengers and the coordinate value of the passenger relative image in the current video sequence.
Preferably, the step S7 specifically includes:
Step S7-1: establishing a data model according to the result of the detection of the passengers in the carriage in the step S5 and the result of the detection of the passengers in the carriage in the step S6 and the detected image sequence;
step S7-2: according to the detection result of the step S5 and the detection result of the step S6, an integrated learning model is established by utilizing a video superposition analysis method;
Step S7-3: and (3) taking the data model established in the step (S7-1) as an input parameter, and detecting passengers by using the integrated learning model to obtain a detection result, wherein the detection result comprises the number of passengers and the relative carriage coordinates of the passengers.
Aiming at the improvement of the existing subway carriage passenger cleaning method, the invention effectively solves the problems of optimizing calculation and transmission processing of the pictures after the video acquisition in the carriage by arranging an interface input/output module, an image analysis and calculation module and an integrated analysis module; the image preprocessing module is arranged to effectively solve the interference of external factors such as equipment shake, illumination, noise and the like on the image, and the image is properly scaled; the problems of loopholes and instability in the analysis of the image through the image analysis and calculation module are effectively solved by combining the image analysis and calculation module with the deep learning analysis and calculation module; and the structure is simple and stable, and the universality is extremely high.
Compared with the prior art, the technical scheme has the following beneficial effects:
1. cross-platform compatibility, support server based on ARM framework of full autonomy and full autonomy video acceleration card;
2. the accuracy of the clear customer identification is improved by adopting the traditional image processing and machine learning dual checking mode;
3. the accuracy of the clear recognition is improved by carrying out targeted modeling based on the clear scene and the carriage video;
4. through regional background modeling and video superposition analysis, accurate judgment of the clear passenger is realized, and detection of the position of the passenger is not finished;
5. the compressed picture is supported to be identified, and the bandwidth limitation in the actual application scene is better compatible;
6. The method and the device for carrying out the clear passenger judgment based on the subway carriage video are complete, and a new method is provided for subway clear passenger.
Detailed Description
The foregoing and other features, aspects and advantages of the present invention will become more apparent from the following detailed description of the embodiments with reference to the accompanying drawings 1 through 2. The following embodiments are described in detail with reference to the drawings.
In a first embodiment, the invention is a device for performing a passenger judgment based on a video of a subway car, comprising an interface input/output module (VCINTERFACE), wherein the interface input/output module (VCINTERFACE) is connected to an ATS interface server outside, a video stream acquired by a video acquisition device is acquired through the ATS interface server, transmission of video or information between the interface input/output module (VCINTERFACE) is implemented through the interface input/output module (VCINTERFACE), the interface input/output module (VCINTERFACE) is responsible for acquiring the video stream from the ATS interface server and converting the video stream into a picture, the picture is transferred to a lower processing module, the output part is responsible for transferring a passenger judgment result to the ATS server, the interface input/output module (VCINTERFACE) is connected to an image analysis calculation module (VCcaculatorA), the video stream is converted into a picture through the interface input/output module (VCINTERFACE) and then transferred to the image analysis calculation module (VCcaculatorA), the image analysis calculation module (VCcaculatorA) is responsible for analyzing the image based on a background and foreground feature extraction method, the passenger analysis module (3992) is connected to the image analysis calculation module (3992) through the integrated image calculation module (integrated with the image calculation module (3896), the integrated analysis module (VCanalyzer) is connected with the interface input/output module (VCINTERFACE), and the integrated analysis result is transmitted to the interface input/output module (VCINTERFACE) through the integrated analysis module (VCanalyzer) and then transmitted to the ATS interface server through the interface input/output module (VCINTERFACE) so as to realize the processing of the video stream in the carriage and the monitoring of the carriage;
The ATS interface server, the interface input/output module (VCINTERFACE), the image analysis and calculation module (VCcaculatorA) and the integrated analysis module (VCanalyzer) are all used for transmitting information and images among the modules;
In the embodiment, firstly, the ATS server collects a video stream and transmits the video stream to the interface input/output module (VCINTERFACE), the video stream converts the video stream to a corresponding picture through the interface input/output module (VCINTERFACE), the corresponding picture is transmitted to the image analysis and calculation module (VCcaculatorA), the image analysis and calculation module (VCcaculatorA) analyzes and calculates the picture, the analysis and calculation result is transmitted to the integrated analysis module (VCanalyzer) after detection, the integrated analysis module (VCanalyzer) performs integrated analysis on the calculation result transmitted by the image analysis and calculation module (VCcaculatorA), and meanwhile, the integrated analysis module (VCanalyzer) transmits the integrated analysis result to the interface input/output module (VCINTERFACE) and then transmits the integrated analysis result to the ATS server to realize the whole processing and analysis of the video stream in the carriage.
In the second embodiment, on the basis of the first embodiment, in the image transmission process, since the shake of the device and the influence of external conditions such as illumination or noise will interfere with the definition of the image and affect the quality of the image, the present embodiment provides a method for eliminating the influence of the external environment on the image, specifically, an image preprocessing module is connected between the interface input/output module (VCINTERFACE) and the image analysis and calculation module (VCcaculatorA), the image transmitted by the interface input/output module (VCINTERFACE) is preprocessed by the image preprocessing module, and after the processing is completed, the processed image is transmitted to the image analysis and calculation module (VCcaculatorA), the image transmitted from the interface input/output module (VCINTERFACE) is preprocessed by the image preprocessing module, so as to eliminate the interference caused by the external factors such as the shake of the device and the illumination or noise on the image, ensure the quality of the processed image, and then the image is properly scaled and then transmitted to the image analysis and calculation module (VCcaculatorA), and the image analysis and calculation module (494) is used for calculating the image.
In the third embodiment, based on the second embodiment, when the number and the position of the passengers are determined only by the image analysis and calculation module (VCcaculatorA), a certain error occurs, which results in a decrease in the positioning accuracy of the passengers, so the present embodiment provides a method for determining the number and the position coordinates of the passengers in a double manner, specifically, the image preprocessing module is connected with the deep learning and analysis and calculation module (VCcaculatorB), the deep learning and analysis and calculation module (VCcaculatorB) and the integrated analysis module (VCanalyzer), the deep learning and analysis and calculation module (VCcaculatorB) is an image calculation module based on a deep learning method, the image preprocessing module performs analysis and calculation by using a deep neural network, the number of passengers in the image preprocessed by the image preprocessing module and the coordinate value of the passenger relative image are acquired, the result and the corresponding image sequence are transmitted to the integrated analysis module (VCanalyzer), the integrated analysis and calculation module (VCcaculatorA) and the integrated analysis and calculation module (VCcaculatorB) are overlapped by using the image, the result and the position of the passenger is accurately transmitted to the passenger interface (35), and the result is accurately transmitted to the passenger interface (35) to perform verification and the position information.
In the fourth embodiment, on the basis of the third embodiment, the image analysis and calculation module (VCcaculatorA) and the deep learning and calculation module are both connected with an autonomous video acceleration card, so that the image analysis and calculation module (VCcaculatorA) and the deep learning and calculation module are completed on the autonomous video acceleration card, and meanwhile, the interface input and output module (VCINTERFACE), the image preprocessing module, the image analysis and calculation module (VCcaculatorA), the deep learning and analysis and calculation module (VCcaculatorB) and the integrated analysis and calculation module (VCanalyzer) are all completed on the basis of an autonomous server of an ARM architecture, and references are provided for the landing application of a domestic platform through the autonomous server of the ARM architecture and the application of the autonomous video acceleration card.
In a fifth embodiment, based on the fourth embodiment, the method for performing passenger clearing judgment based on the subway carriage video includes the following steps:
Step S1: the ATS server accesses the video stream address of the carriage camera to obtain the video stream of the subway carriage picture, the video stream is transmitted to the interface input/output module (VCINTERFACE), and the interface input/output module (VCINTERFACE) converts the video stream into a video picture sequence and transmits the video picture sequence to the image preprocessing module (VCImage processor);
step S2: the image preprocessing module (VCImage processor) is used for carrying out image stabilization analysis on the acquired carriage image sequence, so that the interference on images, which is introduced into the video image sequence due to irregular random motion, is eliminated;
Step S3: performing optical compensation operation on the picture sequence processed in the step S2 through the image preprocessing module (VCImage processor), eliminating the influence of picture color and texture distortion caused by non-ideal white diffuse reflection in the picture, and respectively transmitting picture information to the image analysis and calculation module (VCcaculatorA) and the deep learning analysis and calculation module (VCcaculatorB);
step S4: carrying out image background modeling on the picture sequence processed in the step S3 through the image analysis and calculation module (VCcaculatorA);
Step S5: then, the image analysis and calculation module (VCcaculatorA) carries out passenger detection on the picture sequence modeled in the step S4 and the image sequence processed in the step S3, so as to obtain the number and position coordinate information of the passengers in the picture sequence, and the result information is transmitted to the integrated analysis module (VCanalyzer);
Step S6: carrying out passenger identification based on the deep learning on the picture sequence obtained in the step S3 through the deep learning analysis calculation module (VCcaculatorB), obtaining the number of passengers and position coordinate information in the picture sequence, and transmitting the result information to the integrated analysis module (VCanalyzer);
Step S7: the integrated analysis module (VCanalyzer) is used for carrying out integrated detection on the result of passenger detection in the step S5 and the result of passenger identification in the step S6;
Step S8: when the integrated analysis module (VCanalyzer) is used for carrying out integrated detection, carrying out compartment area division on each compartment of the train, and corresponding a video picture sequence of each camera to each compartment area of the train to establish a mapping relation;
Step S9: detecting the position of a passenger in the carriage according to the region division result of the carriage in the step S8 and the result of the integrated learning in the step S7, transmitting result information to the interface input/output module (VCINTERFACE) through the integrated analysis module (VCanalyzer), and transmitting the result information to the ATS server through the interface input/output module (VCINTERFACE);
Step S10: obtaining the final number of passengers and the positions of the passengers relative to the carriage according to the carriage position detection result in the step S9;
step S11: outputting a detection result according to the step S10, and judging that the passenger clearing is successful if the carriage has no passenger;
step S12: and outputting a detection result according to the step S10, and judging that the passenger clearing is not finished if the passenger exists in the carriage.
In a sixth embodiment, based on the fifth embodiment, in the process of performing analysis and calculation on the preprocessed picture sequence by the deep learning analysis and calculation module (VCcaculatorB), in order to ensure accuracy of data and accuracy of positioning, the present embodiment provides a method for processing the picture sequence by the deep learning analysis and calculation module (VCcaculatorB), specifically, the step S6 is specifically:
step S6-1: establishing a data model of completion of subway carriage clear and incompletion of subway carriage clear in the deep learning analysis calculation module (VCcaculatorB);
Step S6-2: establishing an image recognition model of subway carriage passenger detection based on deep learning, and obtaining parameters of a carriage passenger detection model by utilizing a carriage passenger detection data model established in the step S6-1;
Step S6-3: taking the image sequence obtained in the step S3 as an input parameter, and carrying out passenger detection by utilizing the passenger detection model of the carriage established in the step S6-2 to obtain the number of passengers in the input image sequence and the coordinate value of the relative image of the passengers;
Step S6-4: and (3) judging according to the number of passengers in the image sequence detected in the step S6-3 and the coordinate values of the relative images of the passengers, if no passenger exists in the image detection result, determining that no passenger exists in the passenger identification result based on the deep learning, and if the passenger exists in the image detection result, determining that the passenger exists in the passenger identification result based on the deep learning, wherein the passenger identification result comprises the number of passengers in the current video sequence and the coordinate values of the relative images of the passengers, and then transmitting the identification result to the integrated analysis module (VCanalyzer).
In a seventh embodiment, on the basis of the fifth embodiment, the present embodiment provides a specific method for accurately determining the number and position coordinates of the passengers in the vehicle cabin through the integrated analysis module (VCanalyzer), specifically, the step S7 is specifically:
Step S7-1: establishing a data model according to the result of the detection of the passengers in the carriage in the step S5 and the result of the detection of the passengers in the carriage in the step S6 and the detected image sequence;
step S7-2: according to the detection result of the step S5 and the detection result of the step S6, an integrated learning model is established by utilizing a video superposition analysis method;
Step S7-3: and (3) taking the data model established in the step (S7-1) as an input parameter, and detecting passengers by using the integrated learning model to obtain a detection result, wherein the detection result comprises the number of passengers and the relative carriage coordinates of the passengers.
When the system is specifically used, the ATS interface server, the interface input/output module (VCINTERFACE), the image analysis and calculation module (VCcaculatorA), the deep learning analysis and calculation module (VCcaculatorB) and the integrated analysis module (VCanalyzer) are connected and assembled, and then the system operates according to the following steps:
Step S1, obtaining a video stream through an ATS interface server, transmitting the video stream to an interface input/output module (VCINTERFACE), converting a video picture sequence with a proper size by the interface input/output module (VCINTERFACE), storing the picture in a memory, and then executing step S2;
S2, performing image stabilization processing on an image through an image preprocessing module (VCImageprocessor), performing image stabilization processing on a picture stored in a memory, eliminating interference on the image, which is introduced in a video image sequence due to irregular random motion, and then executing S3;
s3, performing optical compensation on the image in an image preprocessing module (VCImageprocessor), performing optical compensation on the picture processed by the stable pixels, eliminating the influence of picture color and texture distortion caused by non-ideal white diffuse reflection in the picture, and then executing S4;
S4, designing background modeling in an image analysis and calculation module (VCcaculatorA), carrying out background modeling on the optically compensated picture, and then executing S5;
S5, comparing the picture subjected to background modeling with the picture subjected to optical compensation in an image analysis and calculation module (VCcaculatorA) to obtain the foreground characteristics of the passengers in the picture, and then executing S6;
step S6, an image analysis and calculation module (VCcaculatorA) detects the foreground features acquired from the pictures to acquire the information of the passengers in the picture sequence, and then S7 is executed;
step S7, designing a subway carriage passenger detection model based on deep learning in a deep learning analysis calculation module (VCcaculatorB), and then executing S8;
Step S8, designing a picture data model, establishing a picture data model which is suitable for being used as the input of subway carriage passenger detection model data from the picture subjected to optical compensation, and then executing S9;
Step S9, taking the established picture data model as an input parameter, detecting passengers by using a subway carriage passenger detection model, acquiring passenger information in a picture sequence, and then executing S10;
Step S10, the passenger information obtained based on the traditional computer vision in step S6 and the passenger information obtained based on the neural network in step S9 are subjected to integrated learning, the number of passengers in the picture sequence and the coordinate value of the relative picture are obtained, and then S11 is executed;
Step S11, designing compartment area division, corresponding a video picture sequence of each camera to each compartment area of the train, establishing a mapping relation, and then executing S12;
Step S12, designing passenger position detection, carrying out passenger position detection relative to a carriage according to the number of passengers and the coordinate information relative to the picture acquired in the step S10 and the mapping relation between the carriage and the video picture established in the step S11, and then executing the step S13;
and step S13, outputting successful passenger clearing if no passenger exists according to the number and the coordinates of the carriages obtained in the step S12, and outputting the position information of the passenger in the carriage if the passenger exists.
Compared with the prior art, the technical scheme has the following beneficial effects:
1. cross-platform compatibility, support server based on ARM framework of full autonomy and full autonomy video acceleration card;
2. the accuracy of the clear customer identification is improved by adopting the traditional image processing and machine learning dual checking mode;
3. the accuracy of the clear recognition is improved by carrying out targeted modeling based on the clear scene and the carriage video;
4. through regional background modeling and video superposition analysis, accurate judgment of the clear passenger is realized, and detection of the position of the passenger is not finished;
5. the compressed picture is supported to be identified, and the bandwidth limitation in the actual application scene is better compatible;
6. The method and the device for carrying out the clear passenger judgment based on the subway carriage video are complete, and a new method is provided for subway clear passenger.