Summary of the invention
Technical matters to be solved by this invention is, provides a kind of distributed recognition of face track search system, can break through the bottleneck that facial feature database capacity is little and access speed is slow that existing face identification system exists.
Another technical matters to be solved of the present invention is, provides a kind of distributed recognition of face track search method, to break through the bottleneck that facial feature database capacity is little and access speed is slow that existing face identification system exists.
For solving the problems of the technologies described above, the invention provides following technical scheme: a kind of distributed recognition of face track search system, comprising:
Front end unit subsystem, for gathering face characteristic, comprise web camera and face recognition instrument, face recognition instrument is connected with web camera, to obtain video flowing and to carry out decoding and face detection;
Face track data cell server cluster, comprise multiple stage to be connected with face recognition instrument and the face track data cell server of concurrent operation, every platform face track data cell server is built-in with the facial feature database storing the face characteristic data that face recognition instrument is uploaded, and each facial feature database all has the word bank of multiple concurrent operation;
Central server, it is connected with face track data cell server, and search engine service module and interface are set up in its inside, search engine service module comprises search engine pond and aggregation and sorting device, and search engine pond has multiple search engine unit corresponding with face track data cell server; And
Client, docks to sign in central server and realizes business function interface alternation with central server.
Further, between described front end unit subsystem with face track data cell server, face track data cell server is all connected by internet or private network with between central server, between central server with client.
Further, each search engine unit is by a tunnel and the mutual network communication of face track data cell server.
Further, face track data cell server is also built-in with the polymerization and sorting unit that to be polymerized search face characteristic data out and to sort.
On the other hand, the present invention also provides a kind of distributed recognition of face track search method, comprises the steps:
Human face data acquisition step, gathers facial image by the web camera of the front end terminal subsystem being distributed in various places, then is uploaded to face track data cell server by network after carrying out recognition of face process via face recognition instrument;
Face includes step: the face picture that the collection of front end unit subsystem reports is scaled the picture of predetermined dimension by face track data cell server, automatically the word bank possessing idle capacity is searched again, being included by face after characteristic information extraction possesses in the word bank of idle capacity in found, sets up corresponding index information at index database simultaneously;
Face query steps: by the inquiry port of the access visit service interface of client-access face track data cell server, typing search condition, search condition at least comprises face picture and similarity threshold values, then to the concurrent search of all word banks, each word bank returns the front N bar result of maximum similarity, returns to client and check for inquiry after arranging order.
Further, in described human face data acquisition step, TCP communication mode is adopted by face recognition instrument, Active Registration is to face track data cell server, and heartbeat data packets maintenance connection is uploaded in timing, face recognition instrument is reported to face track data cell server after following the tracks of and collecting face picture.
Further, in described human face data acquisition step, the job step of face recognition instrument is as follows:
Step 1: face recognition instrument adopts modularization pattern interconnection network video camera;
Step 2: after obtaining video flowing readjustment by plug-in type access assembly, obtain video flowing, and decoding exports, final decoding output format is RGB24 original bitmap image stream;
Step 3: after obtaining RGB24 original bitmap image stream, carry out face snap;
Step 4: after extracting face picture, is input to the registration communication component of face recognition instrument by picture, upload in face track data cell server cluster to carry out facial image and feature is included.
Further, specifically the comprising the following steps of face query steps:
Step 1: enter face query interface by client, select a picture with face, automatically extract and detect face picture and extract as querying condition parameter, and specify similarity threshold values, then select the mode to scheme to search figure, inquiry request is submitted to the search engine pond of central server;
Step 2: generate in the search engine pond of central server and activate temporary query object, and inquiry request is input to each search engine unit in search engine pond;
Step 3: inquiry request is sent to long-range face track data cell server by tunnel by each search engine unit;
Step 4: after the inquiry request that face track data cell server please receive, face characteristic contrast search is carried out respectively in each word bank, after the polymerization of the face characteristic track record that each word bank returns in face track data cell server inside and sorting unit rearrangement, via the corresponding temporary query object returned to the tunnel of central server communication in the search engine pond of central server;
Step 5: after the temporary query object in central server search engine pond obtains the comparing result that whole face track data cell server returns, then undertaken regrouping after sequence by the polymerization in central server and sorting unit and return to client.
Further, also comprise order video flow step, concrete implementation step is as follows:
Step 1: client is to the web cam device directory listing of central server queries front end;
Step 2: the web camera node choosing arbitrarily to inquire by client, the face recognition instrument getting the front end unit subsystem residing for this web camera registers with it face track data cell server address information be connected;
Step 3: client initiates the order request of a request video real-time streams to relevant face track data cell server, and the packet content of request live video stream at least comprises following field: device coding, user name, password, communication symbol and request command;
Step 4: after face track data cell server receives the video real-time streams request that client sends, after examining coding and user name encrypted message, this socket communication object is inserted in terminal called session channel, then sends by terminal called session channel the instruction that notifies corresponding face recognition instrument is spreaded stream frequently by face track data cell server;
Step 5: receive the video flowing packing entry terminal conversational communication passage reporting the face recognition instrument of the instruction of video flowing will to obtain from web camera, be sent to face track data cell server by the socket communication object of its inside;
Step 6: after face track data cell server receives the video flowing that face recognition instrument reports, is transmitted to all clients of subscribing to video flowing to book server;
Step 7: after client receives the web camera video flowing of the front end unit subsystem that face trajectory unit server forwards is come, by calling the decoding of all kinds of video encoding protocol of the decoding Middleware implementation native system support of native system and playing up display.
Further, in step 3, the order request of described request video real-time streams adopts ICP/IP protocol TCPSocket socket mode to initiate.
After adopting technique scheme, the present invention at least has following beneficial effect: the present invention can under large-scale dedicated network or internet environment, build distributed face track search engine, realize carrying out pipelining face snap by numerous front end unit subsystems be made up of face recognition instrument and web camera in front end, and be submitted to distributed face track data cell server cluster carry out feature extraction with include, then face track inquiry service interface is provided by back-end central server distribution formula search engine service module, realizing by submission one or more face picture is search condition, search for the trace information that this people have passed through which video camera intrasystem, become video in " smart city " construction and identify " large data " important information resource with raw spy, for " safe city ", " sky net ", " anti-terrorism ", in build Information Engineering project such as " large information ", image investigation application provides effect technique means.
Embodiment
It should be noted that, when not conflicting, the embodiment in the application and the feature in embodiment can be combined with each other, and are described in further detail the present invention below in conjunction with the drawings and specific embodiments.
As shown in Figures 1 to 5, the invention provides a kind of distributed recognition of face track search system, comprise front end unit subsystem 1, face track data cell server cluster 2, central server 3 and client 4, wherein, described face track data cell server cluster 2 is gathered by multiple stage face track data cell server 20 to form.
Fig. 1 has set forth general frame and the deployment way of the present invention's distributed recognition of face track search system, can be divided into four-layer structure: namely front end unit subsystem 1 is terminal acquisition layer; Face track data cell server cluster 2 represents the large data resource layer of face track characteristic; Central server 3 represents center search engine and service layer; Client 4 represents application layer.
Described front end unit subsystem 1, for gathering face characteristic, usually can arrange the described front end unit subsystem 1 of many covers, and dispersed placement is in various places, such as: each street, city, traffic intersection, the main occasion such as public place, office building.Described front end unit subsystem 1 specifically comprises web camera 10 and face recognition instrument 12, round-the-clockly when face recognition instrument 12 is flat to connect with web camera 10, and obtain video flowing and the work such as decoding and face detection, and set up terminal called session channel with higher level's face track data cell server 20 and Active Registration to face track data cell server 20.Thus, described front end unit subsystem 1 absorbs facial image by web camera 10 and carries out identifying processing through face recognition instrument 12 again, and the human face data after identifying processing is uploaded to face track data cell server cluster 2 by internet or private network.For promoting the compatibility of face recognition instrument 12 pairs of web cameras 10, each primary video source access assembly can be built-in with in face recognition instrument 12 inside, such as: ONVIF access assembly, Haikang proprietary protocol assembly, large magnificent proprietary protocol assembly, male proprietary protocol assembly advanced in years, extra large core prestige look proprietary protocol assembly, Topsee proprietary protocol assembly, thus, face recognition instrument 12 can matching used web camera 10 can be the product supporting to meet ONVIF specification protocol, also can also adopt the product of the access protocol assembly of other producers.Described face recognition instrument 12 can adopt the processor of the high-performance low-power-consumption of X86-based, also configurable memory storage, such as: solid-state SSD hard disk, to provide storage recording function, can be convenient to relevant video recording when having access to face snap when needing evidence obtaining.
Fig. 2 shows the groundwork functional character of face recognition instrument 12, and its job step is as follows:
Step 1: face recognition instrument 12 adopts modularization pattern interconnection network video camera 10, video source modularization adopts OO implementation method, by setting up basic parent, several conventional Virtual Functions such as connection, disconnection, video flowing callback events are set, example class component object inherits basic parent, heavy duty realizes the relevant Virtual Function method of parent, realizes the plug-in type middleware component characteristic with OO polymorphism;
Step 2: after obtaining video flowing readjustment by plug-in type access assembly, obtain H264 or H265 standard video stream, same realization has the decode component of polymorphism, realize ONVIF access the standard video stream of assembly readjustment or accessed the privately owned decoding video stream output of producer of assembly readjustment by proprietary protocol, final decoding output format is RGB24 original bitmap image stream;
Step 3: after obtaining RGB24 original bitmap image stream, adopt OpenCV increase income storehouse compiling Face datection storehouse carry out face snap, OpenCV is the cross-platform computer vision library based on distribution of increasing income, there is lightweight and efficient feature, achieve a lot of general-purpose algorithms of image procossing and computer vision aspect, the Haar classifier that the present invention adopts OpenCV to provide and based on the human-face detector of LBP feature for face detection, face tracking method is achieved between successive frame, to abandon the face picture of some redundancies, the more effective face picture that reasonable drawing is caught,
Step 4: after extracting face picture, is input to the registration communication component of face recognition instrument 12 by picture, upload in face track data cell server cluster 2 to carry out facial image and feature is included.
Described face track data cell server cluster 2 includes the face track data cell server 20 of multiple stage concurrent operation, as shown in Figure 1, for N platform face track data cell server be sequentially numbered 1 respectively, 2 ... N to be to show difference.Described face track data cell server 20 is responsible for accepting the connection of front end unit subsystem register, face reports, feature is included, Streaming Media forwards service.Every platform face track data cell server 20 is built-in with facial feature database.In the specific implementation, if support fast search, facial feature database capacity must limit to some extent, and be support rapid comparison search, need again to make feature database capacity reach practical size, for this reason, the facial feature database of the present invention to face track data cell server 20 have employed many word banks parallel modes of operation, thus realizes the increase of the hundreds of times of facial feature database capacity in every platform face track data cell server 20.Preferably, the facial feature database of every platform face track data cell server 20 sets up 100 word banks, each word bank deposits 20000 faces, thus make facial feature database of the present invention not only possess high speed access characteristic but also possessed 1,000,000 grades of mass properties, and again by disposing numerous face track data cell server 20, the face characteristic track " large data " having and be worth under battle conditions can be formed, the center search engine service coordinating central server 3 to provide can be supplied to the face track fast query application that user carries out " to scheme to search figure " mode.Face track data cell server 20 is also built-in with polymerization and sorting unit 22 to be polymerized search face characteristic data out and to sort.
Each word bank of the facial feature database in face track data cell server 20 preferably adopts SQlite database, SQLite is a light-duty database, it is the Relational DBMS of observing ACID, it is included in a relatively little C storehouse, compared to this two data base management system (DBMS) of increasing income of Mysql, PostgreSQL, SQlite database processing speed is faster, is suitable for carrying out the application of face matching operation fast.
Search engine service module and interface are set up in described central server 3 inside, and search engine service module mainly comprises search engine pond and aggregation and sorting device 32.Search engine pond has multiple search engine unit 30 corresponding with face track data cell server 20, each search engine unit 30 adopt ICP/IP protocol set up one with the tunnel 31 of the mutual network communication of face track data cell server 20.Described search engine unit 30 is for the treatment of data to provide search service, and polymerization and the Search Results of sorting unit 32 then for providing search engine unit 30 gather and feed back to client 4 again after sorting.
Described client 4 in the place with private network or internet environment, through safety certification can sign in central server 3 afterwards, for achieving a butt joint to obtain the service such as human face data search, video flowing real-time VOD for user and central server 3.Described client 4 can for running the smart terminal product such as computer, smart mobile phone of corresponding client application.
Described face track data cell server 20 can adopt cloud computing mode to dispose, and can be that centralization is disposed, such as, concentrate the cloud computing machine room disposing smart city, also can support distributed deployment, be deployed in many ground cloud computing machine room respectively; Also can also merge with front end unit subsystem 1 and dispose, concrete deployment way is determined by cost-effectiveness requirement and network environment.No matter take which kind of deployment way, each face track data cell server 20 all accepts active connection and the condition managing of central server 3.
When adopting cloud computing mode to dispose face track data cell server 20, the every platform face track data cell server 20 be deployed on cloud computing virtual machine can accept the registration networking of multiple front end unit subsystem 1, distributed face recognition instrument 12 is only undertaken face tracking and is reported with candid photograph and local task of recording a video, and serves as the task that streaming media on demand puts biography video flowing and stream of recording a video is uploaded simultaneously.The feature of this deployment mode is strengthening cloud computing ability, and maintenance management is more convenient.Server handling ability is stronger, and communication on network backbone during inquiry, speed is faster.But cost is higher.
And when face track data cell server 20 is merged deployment with front end unit subsystem 1, described face track data cell server 20 program can be put in the main frame of face recognition instrument 12 and together with identifier program independent operating, when adopting this deployment way, face track data cell server 20 accepts the registration of local face recognition instrument 12, namely serve one to one, the feature of this deployment mode is strengthening distributed terminal computing power, network structure is simpler, namely numerous front end unit subsystems 1 is directly taken in the service of central server 3, differ because front network bandwidth is irregular, some intelligent acess speed is fast, also having is ADSL network, or 4G network, when carrying out distributed search, speed wants slow relative to cloud computing deployment pattern, but advantage does not need to drop into too many pork barrel just can set up large-scale face track search system.
The present invention also provides a kind of distributed recognition of face track search method, and it comprises the following steps:
Human face data acquisition step, gathers facial image by the web camera 10 of the front end terminal subsystem 1 being distributed in various places, then is uploaded to face track data cell server 20 by internet or private network after carrying out recognition of face process via face recognition instrument 12;
Face includes step: front end unit subsystem 1 is gathered the picture that the face picture offered is scaled predetermined dimension by face track data cell server 20, such as: the picture that can be 96 × 96 pixel sizes, again by the typing port of access visit service interface, automatically the word bank possessing idle capacity is searched by face track data cell server 20, being included by face after characteristic information extraction possesses in the word bank of idle capacity in found, set up corresponding index information at index database simultaneously, index information can comprise the time, original image, positional information, the information such as video camera coding and feature record code,
Face query steps: the inquiry port of being accessed the access visit service interface of face track data cell server 20 by client 4, typing search condition, major parameter can comprise: face picture, similarity threshold values etc., then carry out all word banks by access tunnel management object and search for the concurrent search of passage, each word bank returns the front N bar result of maximum similarity, by to be polymerized and sorting unit 32 carries out arranging order, then return to client and check for inquiry.
In the specific implementation, in described human face data acquisition step, TCP communication mode is adopted by face recognition instrument 12, Active Registration is to face track data cell server 20, and heartbeat data packets maintenance connection is uploaded in timing, face recognition instrument 12 is reported to face track data cell server 20 after following the tracks of and collecting face picture.
Specifically comprising the following steps of face query steps:
Step 1: enter face query interface by client 4, select a picture with face, automatic extraction detects face picture and extracts as inquiry essential condition parameter, can support that 1-6 opens face as search condition, specify similarity threshold values, that is: inquiry is more than or equal to the result of this threshold values, then selects the mode to scheme to search figure, inquiry request is submitted to the search engine pond of central server 3;
Step 2: generate in the search engine pond of central server 3 and activate temporary query object, and inquiry request is input to each search engine unit 30 in search engine pond;
Step 3: inquiry request is sent to long-range face track data cell server 20 by tunnel 31 by each search engine unit 30;
Step 4: after the inquiry request that face track data cell server 20 please receive, face characteristic contrast search is carried out respectively in each word bank, the face characteristic track record that each word bank returns after the polymerization of server internal and sorting unit rearrangement, via the corresponding temporary query object returned to the communication tunnel 31 of central server 3 in the search engine pond of central server 3.
Step 5: after the temporary query object in central server 3 search engine pond obtains the comparing result that whole face track data cell server 20 returns, then undertaken regrouping after sequence by the polymerization in central server 3 and sorting unit 32 and return to client 4.
The present invention can also realize the function of the video flowing of web camera 10 in learning on demand front end unit subsystem by client, its concrete implementation step is as follows:
Step 1: client 4 inquires about the web camera 10 facility inventory list of front end to central server 3;
Step 2: client 4 chooses arbitrarily the web camera inquired 10 node, the face recognition instrument 12 getting this front end unit subsystem 1 residing for web camera 10 registers with it face track data cell server 20 address information be connected;
Step 3: client 4 initiates the order request of a request video real-time streams to relevant face track data cell server 20, this request adopts ICP/IP protocol TCPSocket socket mode to initiate, and the packet content of request live video stream contains: the several primary fields of device coding, user name, password, communication symbol, request command;
Step 4: after face track data cell server 20 receives the video real-time streams request that client 4 sends, after examining coding and user name encrypted message, this socket communication object is inserted in terminal called session channel, then sends by terminal called session channel the instruction that notifies corresponding face recognition instrument 12 is spreaded stream frequently by face track data cell server 20;
Step 5: receive the video flowing packing entry terminal conversational communication passage reporting the face recognition instrument 12 of the instruction of video flowing will to obtain from web camera 10, be sent to face track data cell server 20 by the socket communication object of its inside;
Step 6: after face track data cell server 20 receives the video flowing that face recognition instrument 12 reports, is transmitted to all clients 4 of subscribing to video flowing to book server;
Step 7: after client 4 receives web camera 10 video flowing of the front end unit subsystem 1 that face trajectory unit server 20 forwards, by calling the decoding of all kinds of video encoding protocol of the decoding Middleware implementation native system support of native system and playing up display.
Client 4 accounts for the path of the video broadcasting front end unit subsystem 1, also can support following flow process: client → central server → face track data cell server → front end unit subsystem, the actual conditions of subscribing to video routing mode visible network environment are arranged.
Although illustrate and describe embodiments of the invention, for the ordinary skill in the art, be appreciated that and can carry out multiple change, amendment, replacement and modification to these embodiments without departing from the principles and spirit of the present invention, scope of the present invention is limited by claims and equivalency range thereof.