Summary of the invention
In view of this, it is an object of the invention to provide a kind of big market demand plateform system.The gamut realizing that governmental power is run covers, overall process record.
It is an object of the invention to be achieved through the following technical solutions:
The big market demand plateform system of the present invention, including
Data source gathers；
High-speed broadband transmission networks, transmit to memory resource pool for the data that will gather；
Data center's infrastructure cloud, including calculating resource pool, memory resource pool and Internet resources pond, for depositing the Various types of data of collection, calls for rear method, system；
Large data center platform cloud, is used for providing Computational frame and collaboration software, accesses " data ferrum cage " big data platform；
" data ferrum cage " big data platform, including data and recon platform, basic database and thematic data base,
Application layer, accesses " data ferrum cage " big data platform, for providing the market demand submodule for not commensurate, it is achieved the calling, inquire about and manage of data；
Terminal, accesses application layer, is used for reading and calling data.
Further, during described data source gathers, the data of collection include basis and the structural data such as business thematic data, also include image, the unstructured data such as Internet of Things data that video, voice, the Internet daily record, law enforcement equipment gather.
Further, the service that described data center infrastructure cloud provides includes computing basic facility service, storage infrastructure service, network equipment infrastructure services and information safety devices infrastructure services.
Further, described large data center platform cloud provides data base, virtualization, Distributed Calculation, internal memory to calculate, scheme calculating, stream calculation, Cooperative Workflow, development platform service.
Further, the big data analysis digging tool that plateform system adopts, including cluster analysis, association analysis, space-time analysis and qualitative reductive analysis instrument.
Further, by built-in encoding and decoding, transcoding service, the data of different-format are classified as unified WEB can preview format.
Further, disposing firewall system at different security domain boundaries, the superior and the subordinate's network boundary disposes VPN virtual private gateway equipment, disposes IPS intrusion prevention system in core exchange.
The invention has the beneficial effects as follows:
The present invention is built by big market demand plateform system, set up governmental power is run and carry out gamut covering, the big data cloud platform of overall process record, by the collection to derived data multiple in power running, merge, analyze and application, optimize operation flow, distinct power responsibility, supervisory authority runs, record law enforcement sincerity and science evaluation, realize online working, on-line approval, online law enforcement, ensure that power is run recordable everywhere, can disclose, can analyze, realize behavior transparent, program is transparent and supervision is transparent, first-hand scientific basis is provided for government decision.
Other advantages of the present invention, target and feature will be illustrated to a certain extent in the following description, and to a certain extent, will be apparent to those skilled in the art based on to investigating hereafter, or can be instructed from the practice of the present invention.The target of the present invention and other advantages can be realized by description below and claims and obtain.
Detailed description of the invention
Hereinafter with reference to accompanying drawing, the preferred embodiments of the present invention are described in detail.Should be appreciated that preferred embodiment is only for illustrating the present invention, rather than in order to limit the scope of the invention.
The big market demand plateform system of the present invention, including
(1) data source collection: whole platform realizes different business systems, the crawl of different terminals data and convergence by data collection layer 1, forms centralized and unified data resource.These data resources are the key foundation that all functional modules of whole platform are run, and therefore data collection layer is also the basis in whole platform architecture.Convergence platform is docked by data collection layer and different system, obtains upper layer data and processes the various data that application is required, including: the Database Systems of third-party application, various mobile enforcement terminals, task dissemination system, etc.；It also is able to support the acquisition of different types of data, including various Sybases, various types of unstructured data file (such as video and audio, picture, document, etc.)；It also is able to from the New Media such as the Internet, Internet of Things and obtains desired data.
(2) high-speed broadband transmission networks 2: the data for gathering are transmitted to memory resource pool, the network service architectures of whole system platform, comprises high-speed wideband fiber optic network, mobile broadband network, wireless network, satellite network, E-government Intranet, outer net, mobile government net, Internet of Things service etc..
(3) data center's infrastructure cloud 3: include calculating resource pool, memory resource pool and Internet resources pond, for depositing the Various types of data of collection, calls for rear method, system；The service that data center's infrastructure cloud provides includes computing basic facility service, storage infrastructure service, network equipment infrastructure services and information safety devices infrastructure services.
Computing basic facility service mainly includes general basic computing capability, specific extreme scenes computing capability, complex patterns algorithm computing capability.
General basic computing capability includes: provide the computation capability based on file；Computation capability based on internal memory is provided；
Computing capability based on dynamic flow data is provided；Computing capability based on frequent iteration is provided；The Optimizing Queries ability of distributed mass data is provided；Abundant machine learning storehouse and the algorithms most in use storehouse of applicable distributed structure/architecture are provided；Data calculate to be needed to carry out under unified resource scheduling management mechanism；To realize the pool optimum organization of resource.
Specific extreme scenes computing capability includes: for specific extreme service application, there is provided quick, real-time across districts and cities' magnanimity in real time or near-realtime data record combine investigation ability, to ensure that the details of destination object are extracted in upper-layer service application accurately and in time；For similar above scene and similar above service application, it is provided that quick, real-time cross-region magnanimity related data record combine investigation ability, to ensure that the details of relevant factor are extracted in upper-layer service application accurately and in time；
Complex patterns algorithm computing capability includes: provide the quick fulfillment capability of complex patterns or model；The ability using historical data to be trained and assess complex patterns or model is provided；There is provided and use historical data be predicted and continue the ability revised complex patterns or model；
Storage infrastructure service includes: data acquisition: provide (adopting the mode that FTP time delay is uploaded to gather for unstructured datas such as a small amount of picture, documents) such as huge data acquisition (asynchronous to data center for the core data such as HD video, voice, only to record data link in data base), service data acquisition (adopting ETL instrument to carry out data acquisition for system data), unstructured data collections
Network equipment infrastructure services includes: the Internet, e-government Intranet access and Network Load Balance service；
Information safety devices infrastructure services includes: flow cleaning, anti-DDos attack, anti-virus, anti-Trojan, the anti-tamper function of webpage.
(4) large data center platform cloud 4: be used for providing Computational frame and collaboration software, accesses " data ferrum cage " big data platform；Described large data center platform cloud provides data base, virtualization, Distributed Calculation, internal memory to calculate, scheme calculating, stream calculation, Cooperative Workflow, development platform service.
(5) " data ferrum cage " big data platform 5: include data and recon platform, basic database and thematic data base, can realize between basic database interconnecting, can also realizing between thematic data base interconnecting, described basic database includes population-based data base, legal person's basic database, space and geographical basic database and macroeconomy basic database；Described thematic data base includes public security traffic control thematic data base, building thematic data base and civil administration thematic data base.
(6) application layer 6: access " data ferrum cage " big data platform, for providing the market demand submodule for not commensurate, it is achieved the calling, inquire about and manage of data；
Application layer is the specific functional modules of the detailed programs Demand Design according to big data platform and represents effect, utilizes resource and interface, the data that data collection layer convergence is come that data analysis layer provides, carries out processing and showing according to different service logics
(7) terminal 7: access application layer, be used for reading and calling data.Including the Internet, self-aided terminal, mobile phone terminal etc..User can read, by using each Terminal Type, the related data oneself wanted at any time.
In above-mentioned each ingredient, data center's infrastructure cloud, large data center platform cloud, " data ferrum cage " big data platform can become data analysis layer, realize the bottom layer treatment of platform data is worked by data analysis layer, provide strong instrument guarantee for upper-layer functionality module.Bottom layer treatment for data, namely need the hardware resources such as the storage on basis, calculating, network, be also required to system environments and software, therefore can be divided into again architecture layer (various hardware resource), system platform layer, data service layer and application service layer at this layer according to hardware environment environment and process dimension.Realizing the data all management functions in whole life cycle by this Technical Architecture, and provide perfect data model and development interface, the functional module for upper layer application system encapsulates realization rate necessity, perfect.
The big data analysis digging tool that plateform system adopts, including cluster analysis, association analysis, space-time analysis and qualitative reductive analysis instrument.Instrument is to customize based on Hadoop Open Framework and obtains.
By using the system platform of the present invention, it is possible to achieve following functions:
1, data resource combing: help the Various types of data resource related in constituent parts combing our unit's Business Processing and administrative law enforcement；All kinds of forms involved by data resource, file, video library, picture library etc. are carried out combing；Various types of data combing is gone out data producer, Data Source, data content, data generation time, versions of data, flow chart of data processing, data object output etc.；Data resource to be classified according to structural data (database purchase) and unstructured data (file or the storage of other forms)；The metadata of data is carried out combing；The present situations such as data classification, data model, data encoding, data dictionary are carried out combing.
2, big data framework construction: realizing the unified data exchange in area with shared platform is that each service application provides the data exchange of cross-system, shares service, it is achieved each business application system integration in data plane and integrated；Form the data quality management mechanism of " data ferrum cage " business aspect, improve existing core business data by modes such as duplicate removal, amended record, notices；Realize efficient data display, inquiry, analysis, statistics and derivation performance；Progressively realize the data exchange with external system and shared service；Realize the standardized management to centre data, and information resource catalogue can be formed；Big data framework is also the open platform of a sets of data simultaneously, provides data supporting for Third party system, it is achieved the shared, unified of data accesses.Big data framework is by after the excavation of data and analyzing, it may be achieved data data-pushing between each operation system and circulation, thus driving the application between cross-system and linkage.
3, data cloud construction: data cloud makes any service associated with the data can both occur in the position of a centralization, such as data aggregate, data quality management, data cleansing etc., then different systems and user are served data to again, without considering further which data source is these data come from.Data cloud has the services such as data collection, data identification, data process and decision data.
4, data aggregation service: application units are by building big market demand supporting framework around business handling process, the whole process such as individual's law enforcement, department administration behavior are carried out data record, the Deviant Behavior occurred in business and discipline is recorded at any time, analyze at any time, remind at any time, enable individual and department's behavior more comprehensively and objectively to record and to present.
From data gathering tool, pc client (Win, Linux), Web end, mobile terminal (Andriod, IOS), database side and five kinds of capacity gauges of page end, these data may be from computer, mobile phone, flat board or various special equipment (accessing the internet of things equipment of network).
The data collection of PC end and mobile terminal can assigned catalogue and file type automatically, it is possible in real time or timing the data type of needs is grabbed in unified platform automatically；The data collection of WEB terminal, then by manually realizing, oneself selects the data uploaded or the catalogue specified, and the disposable batch of data is uploaded；The crawl of third party database (SQL, ORACLE) internal data is then realized by data base tool (ETL and CEP), the interface of definition standard, data therein and metadata information are grabbed in platform by data dictionary according to data base automatically, accomplishes to write synchronized update with data base by the monitoring in real time of data base's plug-in unit；The capacity gauge of page end is then realized by ripe crawler technology, can capture the related data on the Internet automatically according to instruction and enter plateform system.
Data base gathers adapter and obtains initial data according to the interface type specified and characteristic requirements from different data sources, it is possible to be acquired by modes such as file interface, data base interface, message interfaces, is then standardized processing.Two ways is generally supported in data acquisition: the cycle gathers and instantaneous acquiring.Cycle collection refers to according to different data contents, according to data pick-up cycle, the mode within the time specified, data extracted.Instantaneous acquiring is that system carries out disposable operation at once according to the acquisition condition set, and this action is not repeated after having operated.Instantaneous acquiring is typically used in the data of historical data and Resurvey.Gather adapter and possess visual configuration management ability, the attribute information of collected side metadata is obtained from metadatabase, by graphic interface the collection adapter that different data source capability is different carried out data acquisition, and the scope of data that different data source customizations is gathered and corresponding constraints.
5, data identification service: the present invention is by built-in encoding and decoding, transcoding service, and the data of different-format are classified as unified WEB can preview format, it is simple to unified application.
Full-text search, carries out unified index for all text class data, it is simple to do search key.
Data are classified, and except the dimension of data itself, are used for increasing packing density by the metadata information of unlimited interpolation and two dimensions of data user behaviors log.
Object directory, catalogue is a kind of generic way of taxonomic description data, it is possible to help user quickly to find required data, the structure of general physical directory is removed in object directory service, have employed the mechanism of object to store data, it is possible to arbitrarily sort out data, carry out tissue and retrieval freely.
Content recognition, by the disposal ability of the outer welding system of platform, it is achieved for OCR, audio identification, feature extraction, similarity detection, head portrait identification service ability.
6, data processing service: data processing service provided by the invention includes
Streaming calculates, system detection lower-layer Message, and the real-time calculating by being polymerized, and issues data result to outside, utilizes this outcome procedure, it is possible to Real Time Drive regulation engine realizes corresponding data and processes and data-driven.
Behavior record, data behavior record method is set up for core with each data/file, follow the tracks of the various actions such as the generation of data, migration, copy, lookup, access, change, bonding behavior user and corresponding behavior application, produce the data behavior record daily record of each data/file, can set up the behavior record of data according to the mode of time shaft when the displaying of each data, the safety producing more innovation and application and reinforcement data for us is all brought great convenience by this.
Regulation engine, defines the incidence relation between big data by formulating multiple rule, dispatches each application module by regulation engine, realizes the event-driven application system feedback of automatization.
Stream compression, utilizes intelligent scheduling engine, it is possible to achieve the data various process on backstage and calling, and this scheduling is mainly based upon the scheduling of content information, and common workflow makes a big difference, and is need the scheduling modes based on semantic analysis.
This data intelligence scheduling engine utilizes semantics recognition to realize on privately owned file system basis, and in whole file system, minister responds a monitor process, it is possible to identifies simple semantic, and can do a little related work according to semantic content.
7, data association established model: use unit to extract key index data, deeply searches the internal association between achievement data, and belonging to achievement data the different classes of formulation prevention and control measure of risk, do a good job of it prevention and control and implement；Build up cadre's personal integrity archives and department service restriction model, individual behavior is carried out comprehensively deep description, power enforcement is carried out science and specifically restricts.
Realize the fusion based on incidence relation between system, operation function, service authority, operation flow etc. are carried out combing specification again, the data functions such as time limit relevant to Power Supervision to early warning, tracking, supervisor, feedback etc., authority, identification are embedded work flow process, make identical index can use at different business platform, and between index, have logic association, data can be confirmed mutually, thus embodying the feature of data record and Supervision and Control.
8, decision data: substantial amounts of data produced by operation system and production system and automatization be collected into " data ferrum cage " big data platform, by unified mode by structural data, video, picture, audio frequency, document and other types document classification, management, the information of index is further excavated by recycling data mining.
Index by conventional data standard, based on these indexes, can providing the user the data sheet of various dimension, including user behavior analysis, data mode analysis, storage value analysis etc., these analytical statements may help to user and carry out the correct decisions of necessity.
Platform has got off also by log recording various user behavior and content aware result, and these information contribute to carrying out data-pushing, is automatically used by data Push to user side.
Certainly, the standard API of open platform can integrate with other operation systems, excavates more valuable data analysis, it is provided that more valuable data analysiss.
Fusion by big data platform, set up the analytical model of different dimensions, the data generated in power enforcement, business handling process are carried out intellectual analysis, power operation is carried out indicating risk by various dimensions, analysis is studied and judged, supervision, reach illegal traffic can not circulate, authority intervene whole process record, critical traffic automatic early-warning, and analysis result be recorded in time in personal integrity archives, make individual behavior represent in system comprehensively, reach grasp at any time, in time understanding, constantly prompting, permanent record, it is achieved power operation visualizes, supervision materialization.
What finally illustrate is, above example is only in order to illustrate technical scheme and unrestricted, although the present invention being described in detail with reference to preferred embodiment, it will be understood by those within the art that, technical scheme can be modified or equivalent replacement, without deviating from objective and the scope of the technical program, it all should be encompassed in the middle of scope of the presently claimed invention.