The content of the invention
It is an object of the invention to provide a kind of filter method, filter and the supplying system of stock market data, enterAnd one or more problem caused by limitation and the defect due to correlation technique is at least overcome to a certain extent.
Other characteristics and advantage of the present invention will be apparent from by following detailed description, or partially by the present inventionPractice and acquistion.
According to the first aspect of the invention there is provided a kind of filter method of stock market data, the filter method shouldFor the supplying system of stock market data, the supplying system at least includes providing the first module of stock market data and dividedThe second module of stock market data is sent out, the filter method includes：Obtain the real-time stock quotes from first moduleData；Obtain all stock informations for all subscription terminals subscription that second module is sent；According to the reality gotWhen stock market data and all stock informations, filtration treatment is carried out to the real-time stock quotes data, filteredStock market data afterwards；Stock market data after the filtering are sent to second module, so that second mouldStock market data distribution after the filtering is subscribed to terminal by block to corresponding.
In some embodiments of the invention, based on aforementioned schemes, the real-time stock quotes data are carried out at filteringThe step of reason, including：Detect the generation time of the real-time stock quotes data whether in scheduled duration；Filter out the realityWhen stock market data in generation time exceed the scheduled duration stock market data.
In some embodiments of the invention, based on aforementioned schemes, the real-time stock quotes data are carried out at filteringThe step of reason, in addition to：According to all stock informations, judge in the real-time stock quotes data whether there is with not byThe related stock market data of the stock information of subscription；Filter out the stock with not being subscribed in the real-time stock quotes dataThe related stock market data of information.
In some embodiments of the invention, based on aforementioned schemes, the real-time stock quotes data are carried out at filteringThe step of reason, in addition to：Detect the indices of the real-time stock quotes data compared to previous stock market dataIndices whether change；Filter out in the real-time stock quotes data compared to previous stock market dataThe stock market data that do not change of indices.
In some embodiments of the invention, based on aforementioned schemes, in addition to：Getting from first moduleAfter real-time stock quotes data, by the real-time stock quotes data storage into memory queue.
In some embodiments of the invention, based on aforementioned schemes, obtain described from first module incessantlyReal-time stock quotes data.
In some embodiments of the invention, based on aforementioned schemes, the second module described in periodicity poll, to getState all stock informations of the second module transmission.
According to the second aspect of the invention there is provided a kind of filter of stock market data, the filter shouldFor the supplying system of stock market data, the supplying system at least includes providing the first module of stock market data and dividedThe second module of stock market data is sent out, the filter includes：First acquisition unit, first mould is come from for obtainingThe real-time stock quotes data of block；Second acquisition unit, is subscribed to for obtaining all subscription terminals that second module is sentAll stock informations；Processing unit, for according to the real-time stock quotes data and all stock letters gotThe real-time stock quotes data are carried out filtration treatment, the stock market data after being filtered by breath；Transmitting element, is used forStock market data after the filtering are sent to second module, so that second module is by the stock after the filteringTicket market data distribution subscribes to terminal to corresponding.
According to the third aspect of the invention we there is provided a kind of supplying system of stock market data, including：First module,Second module and the 3rd module, the 3rd module include the filtering dress of the stock market data as described in above-mentioned second aspectPut；Wherein, first module is used to provide real-time stock quotes data；Second module is used to obtain all subscription terminalsAll stock informations subscribed to, and for receiving the stock market data after the filtering that the 3rd module is sent, by the mistakeStock market data distribution after filter subscribes to terminal to corresponding.
In some embodiments of the invention, based on aforementioned schemes, number and first module of the 3rd moduleIn real-time stock quotes data data volume into positive correlation；The number of second module and the 3rd modular filtrationThe data volume of stock market data afterwards into positive correlation, second module number for it is multiple when, multiple described theTwo modules are interconnected to constitute group system.
In the technical scheme that some embodiments of the present invention are provided, by obtaining the real time stock from the first moduleMarket data, and all stock informations for all subscription terminals subscription that the second module is sent are obtained, and then based on above-mentioned real-timeStock market data and above-mentioned all stock informations carry out filtration treatment to real-time stock quotes data so that (above-mentioned 3rd mouldBlock) it need not pay close attention to which terminal have subscribed which stock information, according to the real-time stock quotes data got and need to only ownSubscribe to all stock informations that terminal subscribes to carry out filtration treatment to real-time stock quotes data, and then can effectively improve pairThe filter efficiency of real-time stock quotes data, it is to avoid hash is pushed into the second module and causes transmission delay and loss to haveThe problem of using data.
In addition, by the generation time to real-time stock quotes data, whether have terminal subscribe to, whether be that repeated data is enteredRow judge, stale data, invalid data and repeated data can be effectively filtered out, so can reduce push to the second module or evenThe data volume of terminal is subscribed to, so as to be conducive to improving data transmission efficiency, data transfer delay is reduced, and can also avoidThe problem of using loss of data.
It should be appreciated that the general description of the above and detailed description hereinafter are only exemplary and explanatory, notCan the limitation present invention.
Embodiment
Example embodiment is described more fully with referring now to accompanying drawing.However, example embodiment can be with a variety of shapesFormula is implemented, and is not understood as limited to example set forth herein；On the contrary, thesing embodiments are provided so that the present invention will moreFully and completely, and by the design of example embodiment those skilled in the art is comprehensively conveyed to.
Implement in addition, described feature, structure or characteristic can be combined in any suitable manner one or moreIn example.Embodiments of the invention are fully understood so as to provide there is provided many details in the following description.However,It will be appreciated by persons skilled in the art that technical scheme can be put into practice without one or more in specific detail,Or can be using other methods, constituent element, device, step etc..In other cases, it is not shown in detail or describes known sideMethod, device, realization operate to avoid fuzzy each aspect of the present invention.
Block diagram shown in accompanying drawing is only functional entity, not necessarily must be corresponding with physically separate entity.I.e., it is possible to realize these functional entitys using software form, or realized in one or more hardware modules or integrated circuitThese functional entitys, or realize in heterogeneous networks and/or processor device and/or microcontroller device these functional entitys.
Flow chart shown in accompanying drawing is merely illustrative, it is not necessary to including all contents and operation/step,It is not required to perform by described order.For example, some operation/steps can also be decomposed, and some operation/steps can be closedAnd or part merge, therefore the actual order performed is possible to be changed according to actual conditions.
Fig. 1 diagrammatically illustrates the flow chart of the filter method of stock market data according to an embodiment of the invention, itsIn, the filter method is applied to the supplying system of stock market data, and the supplying system at least includes providing stock market numberAccording to the first module and distribution stock market data the second module.
The filter method of reference picture 1, according to an embodiment of the invention stock market data, including：
Step S102, obtains the real-time stock quotes data from first module.
According to the exemplary embodiment of the present invention, the real-time stock quotes can be obtained from the first module incessantlyData, so ensure that to the second module or even subscribe to the newest stock market data of terminal push.
In an embodiment of the present invention, can after the real-time stock quotes data from first module are gotWith by the real-time stock quotes data storage into memory queue.
By the real-time stock quotes data storage that will get into memory queue, it is ensured that data rate memory,And then the data in the first module can be avoided due to that can not send out and overstock the performance of the first module of excessive influence in time.
Step S104, obtains all stock informations for all subscription terminals subscription that second module is sent.
According to the present invention exemplary embodiment, can the second module described in periodicity poll, to get described secondAll stock informations that module is sent.Subscribed to it is of course also possible to obtain all subscription terminals from the second module in real timeAll stock informations.Embodiments of the present invention are not limited to this.
Step S106, according to the real-time stock quotes data and all stock informations got, to the realityWhen stock market data carry out filtration treatment, the stock market data after being filtered.
For step S106, the filtering scheme that the present invention is proposed be mainly used for filtering stale data, invalid data andRepeated data.It is specific as follows：
Filter stale data：
According to the exemplary embodiment of the present invention, step S106 includes：Detect the generation of the real-time stock quotes dataWhether the time is in scheduled duration；Filter out the stock that generation time in the real-time stock quotes data exceedes the scheduled durationTicket market data.
Filter invalid data：
According to the exemplary embodiment of the present invention, step S106 includes：According to all stock informations, the reality is judgedWhen stock market data in whether there is the stock market data related to not subscribed stock information；Filter out described real-timeThe stock market data related to not subscribed stock information in stock market data.
Filter repeated data：
According to the exemplary embodiment of the present invention, step S106 includes：Detect the items of the real-time stock quotes dataWhether index changes compared to the indices of previous stock market data；Filter out the real-time stock quotes numberThe stock market data not changed compared to the indices of previous stock market data in.
Step S108, the stock market data after the filtering are sent to second module, so that second mouldStock market data distribution after the filtering is subscribed to terminal by block to corresponding.
Correspondingly, embodiments of the invention also proposed a kind of filter of stock market data, similarly, the filteringDevice is applied to the supplying system of stock market data, and the supplying system at least includes the first mould for providing stock market dataSecond module of block and distribution stock market data, it is specific as shown in Figure 2.
The filter 200 of reference picture 2, according to an embodiment of the invention stock market data, including：First obtains singleMember 202, second acquisition unit 204, processing unit 206 and transmitting element 208.
Specifically, first acquisition unit 202 is used to obtain the real-time stock quotes data from first module；SecondAcquiring unit 204 is used for all stock informations for obtaining all subscription terminals subscription that second module is sent；Processing unit206 are used for according to the real-time stock quotes data and all stock informations got, to the real-time stock quotesData carry out filtration treatment, the stock market data after being filtered；Transmitting element 208 is used for the stock row after the filteringFeelings data are sent to second module so that second module by the stock market data distribution after the filtering to correspondingSubscription terminal.
It should be noted that the detail of each module/unit has been in the filter 200 of above-mentioned stock market dataIt is described in detail in the filter method of corresponding stock market data, therefore here is omitted.
Further, embodiments of the invention also proposed a kind of supplying system of stock market data, including：First mouldBlock, the second module and the 3rd module, the 3rd module include the filter 200 of stock market data as shown in Figure 2；Wherein, first module is used to provide real-time stock quotes data；Second module is ordered for obtaining all subscription terminalsAll stock informations read, and for receiving the stock market data after the filtering that the 3rd module is sent, by the filteringStock market data distribution afterwards subscribes to terminal to corresponding.
In some embodiments of the invention, based on aforementioned schemes, number and first module of the 3rd moduleIn real-time stock quotes data data volume into positive correlation；The number of second module and the 3rd modular filtrationThe data volume of stock market data afterwards into positive correlation, second module number for it is multiple when, multiple described theTwo modules are interconnected to constitute group system.
It should be noted that：Each module in the first module and the second module in the above embodiment of the present invention can be onlyIt is made up of, can also be made up of multiple modular units individual module unit.
As the present invention exemplary embodiment, below by Queue module of above-mentioned first module, above-mentioned second module beExemplified by distribution module and AM access module, above-mentioned 3rd module are pushing module, with reference to Fig. 3 to Fig. 5 to technical schemeIt is described in detail：
As shown in figure 3, the supplying system of stock market data according to an embodiment of the invention, including：Queue moduleM100, pushing module M200, distribution module M300 and AM access module M400.Hierarchical structure is used between modules, mould is pushedBlock M200, distribution module M300 and AM access module M400 can be carried out not according to the terminal M500 quantity and stock exchange frequency of accessShut down extending transversely.Wherein terminal M500 can be in smart mobile phone, tablet personal computer, intelligent television and PC etc., but not limited to this,It can be any terminal device with stock exchange function.
Specifically, the supplying system of the stock market data, which includes at least one, is used to receiving and storing the real-time row of stockThe Queue module M100 of feelings data, at least one be used for sorted out from the Queue module with terminal M500 subscribe to stock believeCease matching present quotation data pushing module M200, at least one be used for terminal set up be connected and store terminal subscribe toThe AM access module M400 of stock information；And at least one is used to control pushing module M200 and AM access module M400 stream compressionsDistribution module M300.
In an exemplary embodiment of the present invention, terminal M500 and AM access module M400 can set up TCP(Transmission Control Protocol, transmission control protocol) long connection, and register subscription to AM access module M400Stock information；Distribution module M300 collects the stock that all terminal M500 are subscribed to by way of poll, and is buffered in the moduleIt is interior；The subscription stock set all from distribution module M300 acquisitions of the same timings of pushing module M200；While pushing moduleM200 is uninterrupted to obtain stock present quotation from Queue module M100；Pushing module M200 is according to the present quotation data of acquisitionSet is subscribed to stock to be matched, by the present quotation data-pushing matched with the stock information that terminal is subscribed to distribution moduleM300；Distribution module M300 will receive market data-pushing to other distribution module M300 and AM access module M400；Market numberAccording to finally directly pushing terminal M500 by AM access module M400.
In an embodiment of the present invention, AM access module M400 is only responsible for the stock subscription that storage is connected to the terminal of the moduleInformation, and give terminal M500 by corresponding stock market data-pushing.Wherein, AM access module M400 quantity can be according to terminalM500 quantity is extended.
Distribution module M300 divides administered AM access module M400 by configuring., can be according to user institute in actual useIn region, across the multiple distribution module M300 of the deployment of computer room.By being mutually arranged composition cluster between distribution module M300, carry outData forwarding, is realized unlimited extending transversely.
Meanwhile, distribution module M300 collects all stock subscription information of the convergences of the AM access module M400 in administering, andBy forwarding mutually, final all distribution module M300 hold whole terminal M500 stock subscription information.
Pushing module M200 automatic regular polling distribution modules M300 is collected into all stock subscription information, at the same it is continual fromPresent quotation data are obtained in Queue module M100 high-speed queue, and stock market data are filtered, are then pushed toDistribution module 300.Wherein, when being necessary, can be same by multiple pushing module M200 as shown in the cross arrows in Fig. 3When to same distribution module 300 push market data (different piece) to improve pushing efficiency, otherwise can also be by sameIndividual pushing module M200 pushes (identical or different) market data to multiple distribution modules 300 simultaneously, specifically can be according to needWill be with depending on efficiency, there is no particular restriction.
The stock market data that pushing module M200 only subscribes to terminal are pushed, and maximum does not have using resource is pushedThe transmission of redundant data, also greatlys save flow bandwidth.
In an exemplary embodiment of the present invention, pushing module M200 and Queue module M100 quantity ratio is N:1, whereinN is the integer more than or equal to 1, and as a rule N is more than 2.
Distributed extension under being connected below to magnanimity terminal, and congestion control is carried out in detail under frequent stock exchangeExplanation：
Stock market, which is pushed, has two major features：1st, the stock sum that can subscribe to concern is to limit, and is handed over by real worldEasily existing stock sum is determined；2nd, the stock that magnanimity terminal user synchronization is subscribed to is concentrated in some fancy stocks.
Based on These characteristics, embodiments of the invention use the supplying system of layer architecture.Specifically, magnanimity terminal M500Only communicated with without configuration, high performance AM access module M400, the stock of concern subscription is being attached thereto by terminal M500In AM access module M400.Its characteristic is that AM access module M400 can set up stock in real time according to terminal M500 subscription informationTerminal M500 set is subscribed to, the terminal M500 for paying close attention to identical stock is polymerize, one is used as using such terminal M500 setIndividual push unit carries out market distribution.The each stock that each AM access module M400 can pay close attention to for its access terminal M500Ticket all safeguards such terminal M500 set, and terminal M500 will be offline, send specified communication protocol to be removed from set.It is existingThe total order of magnitude of real world's stock is constant, and the set can be completely stored among AM access module M400 internal memory, usedLinkedHashMap data structures are stored, can quick response magnanimity terminal M500 frequently access, it is offline sameWhen, it can provide quick COLLECTION TRAVERSALSThe efficiency in distribution again.Distribution module M300 has isolated terminal M500, pushing module M200Need to only communicate with distribution module M300 can obtain the quantity of terminal M500 subscription stocks, and pushing module M200 and need not closeSubscription terminal M500 set in heart stock.
Distribution module M300 and AM access module M400 uses communication construction as shown in Figure 4, distribution module M300 and accessModule M400 can set up two ports simultaneously on startup.By using separated port, security can be improved, while canWith bring in higher flexibility, actual deployment can also by two port snoops to different network interface cards, realize in external flux everyFrom.Distribution module M300 is to external port M301 can (HyperText Transfer Protocol Hyper text transfers be assisted using httpView) agreement is easy to docking, and AM access module M400 can use MQTT (Message Queuing Telemetry to external port M401Transport, message queue remote measurement transmission) agreement, save bandwidth and improve efficiency simultaneously.Distribution module M300 is to inner port M302Identical communication protocol is used to inner port M402 with AM access module M400, Binary Serialization agreement can be such as used, calling makesWith the long connection pool modes of TCP, throttling.
The extension of module is similar to the mode that building blocks are built, and a distribution module need to be only disposed when terminal M500 is lessM300 and AM access module M400 can suitably increase AM access module when user increases for the terminal M500 in the whole worldM400, when to other regions, overseas expansion business, can respectively dispose in global each hot spot areas, pass through distribution moduleM300 intercommunication ports-Extending constitutes distributed structure/architecture as shown in Figure 5.
It should be noted that the annexation between distribution module M300 is not limited to the annular connection shown in Fig. 5, also may be usedTo be chain connection, interconnection etc., the implementation of the present invention is not limited to this.
Pushing module M200 is immediately placed in the queue in pushing module M200 after Queue module M100 acquirement data,Queue in pushing module M200 is a memory queue, it is ensured that Queue module M100 will not cause number more slowly due to data transferPerformance is influenceed according to overstocking.
Pushing module M200 can open three circulations on startup, and first is used for being obtained from distribution module M300 for circulationThe stock collection for taking current time access terminal M500 to pay close attention to, which merges, to be buffered in internal memory, and second for circulating from Queue moduleM100 obtains real time data deposit internal queues, and the 3rd is used to circulate the filtering present quotation data from internal queues and is pushed awaySend.
Pushing module M200 first starts the stock collection that circulation obtains all concerns of terminal M500 from distribution module M300Close, its characteristic specifically subscribes to terminal M500 without every stock is got, need to only obtain all connection terminal M500 and closeWhich stock noted.
Pushing module M200 the 3rd starts circulation and obtains market data from internal queues, detects that market data are produced firstWhether the time within t1 seconds (such as 3 seconds), was considered as more than t1 seconds and expired abandons this market data-pushing.If market data do not haveThere is the expired stock set for further detecting whether this stock there are terminal M500 concerns, obtaining first circulation foundation, ifNot in the set, being considered as this stock does not have terminal M500 concerns, abandons this market data-pushing.Market data ifTerminal M500 is paid close attention to, and also needs to judge whether this market data change compared to each index of last market data, such asFruit does not change, and is considered as repeated data and abandons this time pushing.Filtered layer by layer by above-mentioned, stale data, invalid data, repeat numberAccording to being all filtered out, being flowed into distribution module M300 present quotation data will greatly reduce, and mitigate downstream pressure.
Distribution module M300 can open two circulations on startup, and first for circulating from other distribution module M300And the stock collection merging of the AM access module M400 acquisition current time access terminal M500 concerns in administration is buffered in internal memory,It is the same with pushing module M200, terminal M500 is specifically subscribed to without getting every stock, all connection terminals need to be only obtainedWhich stock M500 has paid close attention to, and distribution module M300 can also count the concern terminal M500 sums of every stock currently in addition, useIn data analysis and statistics.
Distribution module M300 can be on startup preset according to current total number of modules in the number of distribution market data per secondLimit, when receiving pushing module M200 propelling data, first judges whether the data received were reached in the unit interval (such as per second)The data upper limit that can be sent, reaches that the upper limit abandons this market data distribution automatically, and records in daily record the miscue.If being not reaching to the processing upper limit, deposit distribution module M300 memory queue.
Second circulation when distribution module M300 starts can obtain from M300 memory queue and cross filter data again,Detection market data are taken out first time and the time difference of deposit queue, think that system is produced if more than t2 seconds (such as 1 second)Congestion abandons this market data distribution, and the miscue is recorded in daily record.Operation maintenance personnel can monitor above-mentioned daily record judgementIt is no to carry out module extension.Last distribution module M300 starts multiple threads and this market data is pushed into connecing in administration simultaneouslyThe distribution module M300 for entering module M400 and mutually extending.
AM access module M400 is stored in internal queues first when receiving distribution module M300 distribution data.AM access moduleM400 on startup can ON cycle, the market data in acquisition queue.When taking out market data, first search this stock and working asWhether preceding AM access module M400 gathers about injection terminal M500, gathers in the absence of terminal M500, without any operation, otherwise opensThread traverses terminal M500 set is opened, market data are pushed to terminal M500 successively.
In the technical scheme of the above embodiment of the present invention, stock market data all employ interior in the circulation of linksArmy's row are decoupled, quick response upstream requests；Meanwhile, expired, repetition, invalid data can be filtered in upstream link,Reduce the circulation of hash；In addition, each layer module can set the disposal ability of itself, it can be carried during beyond the ability upper limitShow that operation maintenance personnel is extended.
Detailed description more than, those skilled in the art are it can be readily appreciated that the technical scheme tool of the embodiment of the present inventionOne or more of have the advantage that：
First, according to certain embodiments of the present invention, the access of millions terminal is supported, terminal can be greatly reduced and connectEnter cost, in the case of the access of magnanimity terminal and high frequently stock exchange, stock market can be also pushed in time；It will close simultaneouslyThe terminal for noting identical stock is polymerize, and is gathered using this as push unit, greatly promotes the pushing efficiency of stock market.
Secondly, data forwarding is carried out by being mutually arranged composition cluster between distribution module, realization is unlimited extending transversely,Meet the demand of high concurrent scene.
Although it should be noted that being referred to some modules or list of the equipment for action executing in above-detailedMember, but this division is not enforceable.In fact, according to the embodiment of the present invention, it is above-described two or moreModule or the feature and function of unit can embody in a module or unit.Conversely, an above-described mouldThe feature and function of block or unit can be further divided into being embodied by multiple modules or unit.
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implementedMode can be realized by software, can also be realized by way of software combines necessary hardware.Therefore, according to the present inventionThe technical scheme of embodiment can be embodied in the form of software product, the software product can be stored in one it is non-volatileProperty storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are to cause a calculatingEquipment (can be personal computer, server, touch control terminal or network equipment etc.) is performed according to embodiment of the present inventionMethod.
Those skilled in the art will readily occur to its of the present invention after considering specification and putting into practice invention disclosed hereinIts embodiment.The application be intended to the present invention any modification, purposes or adaptations, these modifications, purposes orPerson's adaptations follow the general principle of the present invention and including undocumented common knowledge in the art of the inventionOr conventional techniques.Description and embodiments are considered only as exemplary, and true scope and spirit of the invention are by followingClaim is pointed out.
It should be appreciated that the invention is not limited in the precision architecture for being described above and being shown in the drawings, andAnd various modifications and changes can be being carried out without departing from the scope.The scope of the present invention is only limited by appended claim.