Movatterモバイル変換


[0]ホーム

URL:


CN104376087A - Load balance calculation method for distributed database adopting cross backups - Google Patents

Load balance calculation method for distributed database adopting cross backups
Download PDF

Info

Publication number
CN104376087A
CN104376087ACN201410665567.0ACN201410665567ACN104376087ACN 104376087 ACN104376087 ACN 104376087ACN 201410665567 ACN201410665567 ACN 201410665567ACN 104376087 ACN104376087 ACN 104376087A
Authority
CN
China
Prior art keywords
machine
load
machines used
data fragmentation
theoretical optimal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201410665567.0A
Other languages
Chinese (zh)
Other versions
CN104376087B (en
Inventor
宋永智
张学
武新
崔维力
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
TIANJIN NANKAI UNIVERSITY GENERAL DATA TECHNOLOGIES Co Ltd
Original Assignee
TIANJIN NANKAI UNIVERSITY GENERAL DATA TECHNOLOGIES Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by TIANJIN NANKAI UNIVERSITY GENERAL DATA TECHNOLOGIES Co LtdfiledCriticalTIANJIN NANKAI UNIVERSITY GENERAL DATA TECHNOLOGIES Co Ltd
Priority to CN201410665567.0ApriorityCriticalpatent/CN104376087B/en
Publication of CN104376087ApublicationCriticalpatent/CN104376087A/en
Application grantedgrantedCritical
Publication of CN104376087BpublicationCriticalpatent/CN104376087B/en
Activelegal-statusCriticalCurrent
Anticipated expirationlegal-statusCritical

Links

Classifications

Landscapes

Abstract

The invention relates to a load balance calculation method for a distributed database adopting cross backups. The method includes the steps that load balance of machines in a cluster is ensured when no machine in the initial cluster is damaged; after one machine is damaged, a request distribution mechanism is adjusted properly to ensure the load balance of the machines in the cluster; after several machines are damaged, if all data fragments ensure that more than one machine is available, the request distribution mechanism is adjusted properly, and the load balance of the machines in the cluster is ensured to the greatest extent. The load balance of the distributed database adopting cross backups can be ensured from the macroscopic angle and the angles of probability and coarseness, reasonable equally-shared load pressure of available machines is ensured particularly when certain machines are damaged, the situation that as certain machines are too high in load, more machines are damaged within a short period is avoided, and the problem of the cask effect is avoided.

Description

A kind of computing method adopting the distributed data base load balancing of intersection backup
Technical field
The invention belongs to distributed data base technique field, especially relate to a kind of computing method adopting the distributed data base load balancing of intersection backup.
Background technology
Now, in the face of the pressure of mass data storage and process, unit database cannot bear this important task.And reach its maturity along with distributed computing technology develops, distributed data base has become the main trend of industry.Present distributed data base adopts multinode data base concurrency mode of operation mostly, as GreenPlum, Vertica etc., mass data is stored into each node in cluster by certain Distribution Algorithm, the data volume that each back end is deposited is unlikely to too large, significantly can shorten the execution time when performing complicated inquiry, and internodal parallel processing is the powerful measure improving overall query performance.The natural advantage that distributed data base shows in the field such as mass data storage, calculating causes academia and industry member is more and more paid close attention to.
Distributed data base system is that the basis based on single-node data storehouse system grows up, and is the product that computer technology, data storage technology and network technology combine.For the consideration of high-performance, high reliability and high scalability, distributed data base system needs to carry out data fragmentation to database, and usually needs the redundant data increasing some.Redundant data can waste a lot of storage space; and easily cause the inconsistency between each backup burst; in addition many data fragmentations also can bring the problem of load balancing of machine; optimal load equilibrium is difficult to reach; usually the access pressure of some burst can be caused in actual applications excessive; cause machine breakdown, cause the short slab problem of data-base cluster.
Generally all can carry out burst to data in distributed data base, each data fragmentation can have some redundant slices, in case burst is unavailable or machine is unavailable.In the face of high concurrent, the business of high access, needs to ensure that distributed data base system can provide service lastingly, efficiently.Data fragmentation damage and machine breakdown, unavailable be unavoidable, all the more so time particularly the load of certain machine is extra high.Therefore, how proof load is balanced, makes load share equally on all available machines, and the machine access pressure reducing load higher just becomes particularly important.When especially having machine breakdown in distributed experiment & measurement system, if well do not adjust the request access pressure of each available machines used, so probably other machines because the access of high concurrent large pressure also can be very fast damage.
The granularity of the data fragmentation of distributed data base can be the data block of certain fixed size, can be table rank, even also can be database level other, decide according to the application of database and concrete implementation.The same deployment way disposing ground data fragmentation mainly contains between machine standby mutually, and intersection backup etc. deployment way between multimachine device, such as GreenPlum is exactly the mode backed up of intersecting between multimachine device.
In order to ensure the load balancing of distributed data base, can request be mail in the available machines used of the data fragmentation corresponding to this request in the mode of the mode of Round Robin or random selecting, but once after having machine breakdown in data-base cluster, there is very large drawback in these modes, a certain machine loading may be caused overweight, form short slab, cause the damage of more multimachine device in the short time.
Summary of the invention
For the problems referred to above, the problem to be solved in the present invention is to provide a kind of computing method adopting the distributed data base load balancing of intersection backup, mainly to solve problem of load balancing in data-base cluster, particularly solves problem of load balancing when having machine breakdown.
Core concept of the present invention is: initial cluster is without ensureing during machine breakdown that in cluster, each machine loading is balanced; After having 1 machine breakdown, balanced to each machine loading in Requests routing mechanism suitably adjustment guarantee cluster; After having multiple stage machine breakdown, if all data fragmentations all ensure when being greater than 1 available machines used, suitable adjustment is done to Requests routing machine, ensure that in cluster, each machine loading is balanced to greatest extent.
For solving the problems of the technologies described above, the technical solution used in the present invention is:
Adopt computing method for the distributed data base load balancing of intersection backup, comprise:
When without machine breakdown, ensure that issued request is shared equally in each available machines used at data fragmentation place in the mode of dividing equally from probability; Concrete, if in cluster to the request that certain data fragmentation conducts interviews, get and n is designated as to the available machines used number of the request of this data fragmentation, then the access of this request all born by n platform machine with the probability of 1/n, when visit capacity is enough large time, can be balanced from the angle proof load of probability, such advantage is almost not introduce any extra calculation cost;
When there being 1 machine breakdown, because distributed data base at least can have 1 extra backup burst to each data fragmentation, so data-base cluster now or available; For this situation, the pressure that cluster Requests routing load-balancing method shares request equally from all data fragmentations rapidly switches to non-mode of sharing equally; Need the theoretical optimal load first calculating each machine, if the request access issued is to when damaging the data fragmentation of machine, need to be born by the available machines used that this data fragmentation is corresponding, for each data fragmentation, utilize the theoretical optimal load value of available machines used and current load value to calculate the request of this burst with which type of probability assignments to each available machines used, final ensure all current can Work machine load balancing;
When have be greater than 1 machine breakdown time, because each data fragmentation of distributed data base at least can have 1 extra backup burst, the disabled risk of whole data-base cluster (such as when each data fragmentation all only has 1 to back up burst, and when the machine at these two data fragmentation places all damages) is there is so have when being more than or equal to 2 machine breakdowns; In this case, only have when data-base cluster can be used, time also namely each data fragmentation at least retains an available machines used, each available machines used load balancing as much as possible can be ensured by the method for probability; Need the theoretical optimal load first calculating every platform machine, if the request access issued is to when damaging the data fragmentation of machine, need all to be born by the backup burst in available machines used corresponding to this burst, if the theoretical optimal load value that had the load of machine to exceed, need to recalculate theoretical optimal load value, then the utilize theoretical optimal load value of available machines used and the current load value of burst to calculate the request of this burst with which type of probability assignments to each available machines used one by one, final ensure all can Work machine load balancing.
Preferably, when there being 1 machine breakdown, if the available machines used of data fragmentation of request to comprise when damaging machine will with the strategy divided equally by this request dispatching in current available machine; Utilize theoretical optimal load value and machine current load value to calculate each available machines used with the strategy of greed if the available machines used of the data fragmentation of request does not comprise when damaging machine, make request with different probability assignments to different machines;
In the calculating of probability all the time in system the request access load of each data fragmentation identical premised on, be every platform machine loading upper limit with the theoretical optimal load value of machine, calculate time with probability calculation for mainly to calculate means, extra burden is not added to system.
Preferably, when there being 1 machine breakdown, from damaging the data fragmentation that comprises of machine as point of penetration, calculate its request dispatching to the probability of each available machines used; The load calculating other data fragmentations stored on the non-damaging machine damaging data fragmentation on machine afterwards distributes, and calculates the probability that it distributes to each available machines used; The load finally calculating the data fragmentation do not stored on the non-damaging machine damaging data fragmentation on machine distributes, and calculates the probability that it distributes to each available machines used.
Preferably, when have be greater than 1 machine breakdown and data-base cluster is in upstate time, need the theoretical optimal load first calculating every platform machine;
If the available machines used of data fragmentation of request to comprise when damaging machine will with the strategy of the strategy divided equally and greed by this request dispatching in current available machine;
If the load of certain machine has exceeded theoretical optimal load, again to carry out Adjustable calculation to theoretical optimal load, increase the theoretical optimal load value of other machines, and the theoretical optimal load after Use Adjustment carry out after calculating;
Theoretical optimal load value and machine current load value is utilized to calculate if the available machines used of the data fragmentation of request does not comprise when damaging machine, make request with different probability assignments to different machines, to ensure that the load of each machine is balanced as far as possible, and all level off to theoretical optimal load value;
All load Distribution Calculation are all only comprise probability calculation, extra burden is not had to system, in the calculation all the time in system the request access load of each data fragmentation identical as far as possible premised on, be every platform machine loading upper limit with the theoretical optimal load value of machine.
Preferably, when have be greater than 1 machine breakdown and data-base cluster is in upstate time, first process damages the data fragmentation stored in machine, each data fragmentation is processed from less to more according to the available machines used number of correspondence, namely the preferential load calculating the few data fragmentation of available machines used distributes, and calculates its request dispatching to the probability of each available machines used; Secondly process other data fragmentation, utilize the theoretical optimal load value of every platform machine and current load value to carry out probability calculation to request, determined the probability distributing to each available machines used.
Preferably, if the load of certain machine has exceeded theoretical optimal load, again will carry out Adjustable calculation to theoretical optimal load, increase the theoretical optimal load value of other machines, the algorithm of adjustment is: the proportion setting the theoretical optimal load value of every platform machine to account for total load aswherein: the machine number in cluster is MAX, the machine number of damage is catwalk, then remaining available machines used number is MAX-T; The proportion that the rear optimal load of adjustment accounts for total load iswherein: the machine exceeding theoretical optimal load is Mq, and its current load value is Tm, Tm is deducted from theoretical optimal load calculates, if the set of adjustment available machines used is E={Mi| Mi∈ (B-Mq) }, damage the available machines used set corresponding to each burst of the data fragmentation in collection of machines D
The advantage that the present invention has and good effect are:
With solution of the prior art, as ostrich algorithm is ignored this problem, suspends the access of this machine when certain machine pressure is large, carried out storages Data Migration and reduce single point pressure etc. and compare, the inventive method can not cause single machine pressure excessive to such an extent as to causing trouble generation; Can not suspend the access to database, guarantee business is normally run; Be applicable to the various distributed data base adopting intersection backup mode; The overhead brought system is very little, only has the probability calculation of some comparison basis; Can adjust etc. system pressure at any time;
The inventive method can load distribution in the adjustment cluster of coarseness, is that Main Means controls each request and distributes the load of each available machines used, can be good at ensureing that in cluster, each machine loading is balanced, avoids the generation of short slab problem with probability.
Accompanying drawing explanation
Fig. 1: one embodiment of the invention distributed data base intersection back-up storage schematic diagram;
Fig. 2: one embodiment of the invention distributed data base intersects back-up storage without load balancing schematic diagram during machine breakdown;
Fig. 3: one embodiment of the invention distributed data base intersection back-up storage, single machine damage plan;
Fig. 4: one embodiment of the invention distributed data base intersection back-up storage, single machine damages load balancing calculation flow chart;
Fig. 5: one embodiment of the invention single machine damages, damages data fragmentation load balancing calculation flow chart in machine;
Fig. 6: one embodiment of the invention single machine damages, data fragmentation load balancing calculation flow chart in non-damaging machine;
Fig. 7: one embodiment of the invention distributed data base intersection back-up storage, more than 1 machine breakdown schematic diagram;
Fig. 8: one embodiment of the invention distributed data base intersection back-up storage, more than 1 machine breakdown load balancing calculation flow chart;
Fig. 9: one embodiment of the invention, more than 1 machine breakdown, damages data fragmentation load balancing calculation flow chart in machine;
Figure 10: one embodiment of the invention more than 1 machine breakdown, data fragmentation load balancing calculation flow chart in non-damaging machine.
Embodiment
For enabling above-mentioned purpose of the present invention, feature and advantage become apparent more, and below in conjunction with the drawings and specific embodiments, the present invention will be further described in detail.
Consult Fig. 1, the data-base cluster example adopting intersection backup mode to store data fragmentation is shown in the present invention, has 6 machines in this cluster, totally 6 data fragmentations, each burst all have 1 back up burst, these bursts with intersects back up mode spread in each machine of data-base cluster.
Consult Fig. 2 without during machine breakdown, system normal condition in the present invention is shown, without being divide equally user to ask the mode of data fragmentation place machine to ensure system load balancing during machine breakdown in data-base cluster.If the machine number in cluster is MAX, then the machine in data-base cluster can be expressed as Mi(1≤i≤MAX).Each burst corresponding can be expressed as Nj(1≤j≤MAX).If the initial pressure value of every platform machine is set to Q (Mi)=0, the theoretical optimal load value (force value) of every platform machine is(accounting for the proportion of total load), for the too machine of 6 shown in Fig. 1, the force value of each machine is 1/6th.For the request of each data fragmentation, be sent in the available machines used of this burst by it in the mode of dividing equally, can realize the load balancing of coarseness from the angle of probability, the number percent that the load that every platform machine is born accounts for total load is
When having 1 machine breakdown, because each data fragmentation of distributed data base at least can have 1 extra backup burst, so when only having 1 machine breakdown, whole data-base cluster must be available.
Consult Fig. 3, illustrate in the present invention that when having 1 machine breakdown, the state of data-base cluster, for machine M2 damages in the present embodiment wherein.If the machine number damaged is t, in data-base cluster, machine number becomes MAX-1, and the machine in data-base cluster can be expressed as Mi(1≤i≤MAX & & i ≠ t).
Consult Fig. 4, when 1 machine breakdown in the present invention is shown, data-base cluster is done to the flow process of load balancing calculating.First data fragmentation is divided into groups, be divided into the data fragmentation stored in the data fragmentation and non-damaging machine damaging and store in machine; Processing from less to more according to the available machines used number damaging the data fragmentation stored in machine, the preferential load calculating the few data fragmentation of available machines used, with Greedy strategy? ensure that the load of each machine is balanced as much as possible.If the initial pressure value Q (M of every platform machinei)=0, the proportion that the theoretical optimal load value of every platform machine accounts for total load iswhen larger to the visit capacity of database, can think that to the access probability of each burst be the same, namely the access load of each burst is
Consult Fig. 5, when 1 machine breakdown in the present invention is shown, to the flow process that the data fragmentation damaged in machine processes.Because the available machines used number of the data fragmentation in damage machine is less than the data fragmentation in non-damaging machine, the load therefore wanting priority processing to damage the data fragmentation machine from the angle of Greedy strategy calculates.Concrete steps are as described below:
Step S501, data fragmentation is divided into two groups, namely damages data fragmentation that machine stores and the data fragmentation that normal machines stores, can draw and damage machine Mtdata fragmentation set be A={Nx| 1≤x≤MAX & & x ∈ Mt, the available machines used set B={ M corresponding to each burst in normal machinesi| 1≤i≤MAX & & i unequal to t}.Each burst in set A is different according to backup copies number, may correspond to one or some machines, such as, correspond to 1 machine when 2 burst exactly, and correspond to 2 machines when 3 burst exactly.Process each burst from less to more according to the available machines used number of correspondence, namely first process correspond to those bursts of 1 machine, and then then process correspond to those bursts of 2 machines, the like.Obtain Mtin a certain data fragmentation, for only having the data fragmentation order of 1 available machines used to perform step S502, otherwise jump to step S503.
Step S502, only have the data fragmentation of 1 available machines used, be set to Nt, its request access pressure is born by this separate unit available machines used 100%, if this machine number is j, then upgrades the load value Q (M of this machinej)=Q (Mj)+P (Nt) * 100%.Finally, step S506 is jumped to.
Step S503, for there being the machinable data fragmentation of multiple stage, then should to bear by all machines in the available machines used set B corresponding to this burst; The access load of burst each in set A isthe mode that the existing load of the machine that this burst is corresponding adds to share equally is assigned to the load of this machine (just for calculating, do not distribute in the mode of sharing equally) the theoretical optimal load value that do not exceed every platform machine when accounting for the proportion Tq of total load order perform step S504, if exceeded Tq, jumped to step S505.
If this burst of step S504 (is set to burst Nt) load of corresponding machine not more than Tq, then to the probability that the access of this burst is assigned to corresponding machine in set B in the mode of sharing equally beupgrade the load value Q (M of corresponding machinei)=Q (Mi)+P (Nt) * P (Mi| Mi∈ B), finally jump to step S506;
If this burst of step S505 (is set to burst Nt) some corresponding machine loading mode of adding to share equally is assigned to the load (just for calculating, not distributing in the mode of sharing equally) of this machine more than Tq, if the load all assigned to is Qavg(being greater than Tq), if the set of these machines is BM={Mi, 1≤i≤MAX & & i unequal to t & & Q (Mi)+Qavg>=Tq}, i.e. set A and the set B machine that has data fragmentation to occur simultaneously; To the request of each data fragmentation, obtain the list of its available machines used, preferentially calculate the machine loading be in set B M, namely this request dispatching to the probability of certain machine in BM set isupgrade the load value Q (M of corresponding machinei)=Q (Mi)+P (Nt) * PB, the load of these machines can be made like this can not to exceed theoretical optimal load, and make the load of each machine balanced as much as possible, shared is to the access pressure of stored data fragmentation.
Step S506, judge whether to process the data fragmentation in all set A, if not, jump to step S501.
Consult Fig. 6, when 1 machine breakdown in the present invention is shown, to the flow process that the data fragmentation in non-damaging machine processes.Because occupied the load of available machines used after completing the data fragmentation process damaged in machine, so theoretical optimal load and the present load of machine will be considered when carrying out load calculating for the data fragmentation of non-damaging machine, make the load of each machine theoretical optimal load of convergence as far as possible.
Step S601, after completing and carrying out load balancing calculating to the data fragmentation in set A, be designated as G for its number of the burst in non-set A, its total access load isbelonging to these bursts, machine is known, is set to set C={Mj| 1≤j≤MAX & & j unequal to t} is for each burst in non-set A, and the load balancing of carrying out following step calculates:
Step S602, the data fragmentation traveled through in non-set A, find the data fragmentation N not yet completing load and calculatet, and the available machines used of this data fragmentation all can find in set C and set B, i.e. there is the available machines used M of this burst in the common factor of set B and Cp;
Step S603, (burst N is set to this data fragmentationt) request access distribute to set B and the machine M that occurs simultaneously of set Cpprobability bedistribute to other a certain machines (non-machine M in corresponding set Cp, be set to Ms) probability beupgrade Q (Mp)=Q (Mp)+P (Nt) * PB, this machine is moved from set C and is put into set B; Upgrade Q (Ms)=Q (Ms)+P (Nt) * PC;
Step S604, the step constantly repeating above, until process the data fragmentation in all non-set A.Can ensure that all available machines used share the request load of user equally, ensure that the short-board effect not having single point pressure to cause more greatly occurs.Concerning each data fragmentation, be all with different probability assignments give each available machine, there will not be the situation that cannot respond the request of a certain data fragmentation.
Consult Fig. 7, illustrate in the present invention and have when being greater than 1 machine breakdown, and data-base cluster is in upstate, the machine damaged in Fig. 7 is M2 and M4, but data-base cluster is in upstate.If the machine damaged has catwalk, the set damaging machine is D, and the indexed set damaging machine is S={1≤i≤MAX, Mi∈ D}, in data-base cluster, machine number becomes MAX-T, and the machine in data-base cluster can be expressed asonly illustrate in Fig. 7 that each data fragmentation has 1 situation backing up burst, in fact can have multiple backup burst, backup burst is more, and the ability of tolerance machine breakdown is stronger.
Consult Fig. 8, when to illustrate in the present invention more than 1 machine breakdown, and data-base cluster be in upstate, the flow process that load balancing calculates is done to data-base cluster:
First data fragmentation is divided into groups, be divided into the data fragmentation stored in the data fragmentation and non-damaging machine damaging and store in machine; Process from less to more according to the available machines used number damaging the data fragmentation stored in machine, with Greedy strategy, the preferential load calculating the few data fragmentation of available machines used, ensures that the load of each machine is balanced as much as possible.If the initial load value Q (M of every platform machinei)=0, the proportion that the theoretical optimal load value of every platform machine accounts for total load iswhen larger to the visit capacity of database, can think that to the access probability of each burst be the same, namely the access load of each burst isP(Nj)=100MAX.
Consult Fig. 9, when to illustrate in the present invention more than 1 machine breakdown, the data fragmentation damaged in machine is carried out to the flow process of load balancing calculating: reason is because the available machines used number damaging the data fragmentation in machine is less than the data fragmentation in non-damaging machine equally, wants the load of these data fragmentations of priority processing to calculate from the angle of Greedy strategy.Concrete steps are as described below:
Step S901, data fragmentation is divided into two groups, namely damages data fragmentation that machine stores and the data fragmentation that available machines used stores, can show that the data fragmentation set damaged in collection of machines D is set to A={Nx| 1≤x≤MAX & & x ∈ S}, the available machines used set corresponding to each burst also in known seteach burst in set A is different according to number of copies, several machines may be correspond to, each burst is processed according to the number mode from less to more of corresponding machine, namely first process correspond to those bursts of 1 machine, then then process correspond to those bursts of 2 machines, the like.Obtaining a certain data fragmentation damaging and damage machine in collection of machines D, for only having the data fragmentation order of 1 available machines used to perform step S902, otherwise jumping to step S903.
Step S902, only have the data fragmentation of 1 available machines used, be set to Nt,its request access pressure is born by this separate unit available machines used 100%, if this machine number is j, then upgrades the load value Q (M of this machinej)=Q (Mj)+P (Nt) * 100%.Finally, step S906 is jumped to.
Step S903, for the machinable data fragmentation of multiple stage, should to bear by the machine in the available machines used set B corresponding to this burst.In set A, the access load of each burst isthe load (just for calculating, not distributing in the mode of sharing equally) that the mode that the existing load of the machine that this burst is corresponding adds to share equally is assigned to this machine, not more than execution sequence step S904 during Tq, if exceeded Tq, has jumped to step S905.
If this burst of step S904 (is set to burst Nt) load of corresponding machine not more than Tq, then to the probability that the access of this burst is assigned to corresponding machine in set B in the mode of sharing equally beupgrade the load value Q (M of corresponding machinei)=Q (Mi)+P (Nt) * P (Mi| Mi∈ B).Then judge whether the present load of the machine bearing request load exceedes theoretical optimal load value, if exceed theoretical optimal load value, jumps to step S906, otherwise jumps to step S907.
If this burst of step S905 (is set to burst Nt) some corresponding machine loading mode of adding to share equally is assigned to the load (just for calculating, not distributing in the mode of sharing equally) of this machine more than Tq, if the load all assigned to is Qavgif the set of these machines isto the request of each data fragmentation, obtain the list of its available machines used, preferentially calculate the machine loading be in set B M, namely request dispatching to the probability of certain machine in BM set isupgrade the load value Q (M of corresponding machinei)=Q (Mi)+P (Nt) * PB.The load of these machines can be made like this can not to exceed theoretical optimal load, and ensure that the load of each machine is balanced as much as possible.
Step S906, because damage machine more, cause the pressure of some data fragmentation can only be born by same machine, its load cannot be shared by other machines.Judge whether that the load of machine exceedes theoretical optimal load value here, the load if any machine has exceeded theoretical optimal load value and has then carried out calibration calculations to theoretical optimal load, if do not exceed theoretical optimal load, jumps to step S907.If the machine exceeding theoretical optimal load is its current load value of Mq is Tm, Tm is deducted from theoretical optimal load calculates, if the set of calibration available machines used is E={Mi| Mi∈ (B-Mq), then calibrating the proportion that optimal load accounts for total load isall use calibration optimal load as theoretical optimal load in calculating afterwards.
Step S907, the step constantly repeating above, until process the data fragmentation in all set A, ensure that between each machine, load is balanced as far as possible like this.
Consult Figure 10, when to illustrate in the present invention more than 1 machine breakdown, to the flow process that the data fragmentation in non-damaging machine processes.Because occupied the load of available machines used after completing the data fragmentation process damaged in machine, so theoretical optimal load (may be calibration optimal load) and the present load of machine will be considered when carrying out load calculating for the data fragmentation of non-damaging machine, the load of balanced every platform machine distributes, and makes the load of every platform machine all reach optimum.
Step S1001, after completing and carrying out load balancing calculating to the data fragmentation in set A, be designated as G for its number of the burst in non-set A, its total access load isbelonging to these bursts, machine is known, is set to setfor each burst in non-set A, the load balancing of carrying out following step calculates.
Step S1002, the data fragmentation traveled through in non-set A, find the data fragmentation not yet completing load and calculate, and the available machines used of this data fragmentation all can find in set C and set B, be i.e. there is available machines used M in the common factor of set B and Ct
Step S1003, (burst N is set to this data fragmentationt) request access distribute to set B and the machine M that occurs simultaneously of set Ctprobability bedistribute to other a certain machines (non-machine M in corresponding set Ctbe set to Ms) probability beupgrade Q (Mp)=Q (Mp)+P (Nt) * PB, this machine is moved from set C and is put into set B.Upgrade Q (Ms)=Q (Ms)+P (Nt) * PC.
Step S1004, the step constantly repeating above, until process the data fragmentation in all non-set A.Can ensure that the load of all available machines used is balanced as far as possible, the request load of shared user.
Above embodiments of the invention have been described in detail, but described content being only preferred embodiment of the present invention, can not being considered to for limiting practical range of the present invention.All equalizations done according to the present patent application scope change and improve, and all should still belong within patent covering scope of the present invention.

Claims (6)

3. the computing method adopting the distributed data base load balancing of intersection backup according to claim 2, it is characterized in that: when there being 1 machine breakdown, from damaging the data fragmentation that comprises of machine as point of penetration, calculate its request dispatching to the probability of each available machines used; The load calculating other data fragmentations stored on the non-damaging machine damaging data fragmentation on machine afterwards distributes, and calculates the probability that it distributes to each available machines used; The load finally calculating the data fragmentation do not stored on the non-damaging machine damaging data fragmentation on machine distributes, and calculates the probability that it distributes to each available machines used.
5. the computing method adopting the distributed data base load balancing of intersection backup according to claim 4, it is characterized in that: when have be greater than 1 machine breakdown and data-base cluster is in upstate time, first process damages the data fragmentation stored in machine, each data fragmentation is processed from less to more according to the available machines used number of correspondence, namely the preferential load calculating the few data fragmentation of available machines used distributes, and calculates its request dispatching to the probability of each available machines used; Secondly process other data fragmentation, utilize the theoretical optimal load value of every platform machine and current load value to carry out probability calculation to request, determined the probability distributing to each available machines used.
6. the computing method adopting the distributed data base load balancing of intersection backup according to claim 4, it is characterized in that: if the load of certain machine has exceeded theoretical optimal load, again will carry out Adjustable calculation to theoretical optimal load, increase the theoretical optimal load value of other machines, the algorithm of adjustment is: the proportion setting the theoretical optimal load value of every platform machine to account for total load aswherein: the machine number in cluster is MAX, the machine number of damage is catwalk, then remaining available machines used number is MAX-T; The proportion that the rear optimal load of adjustment accounts for total load iswherein: the machine exceeding theoretical optimal load is Mq, and its current load value is Tm, Tm is deducted from theoretical optimal load calculates, if the set of adjustment available machines used is E={Mi| Mi∈ (B-Mq), damage the available machines used set corresponding to each burst of the data fragmentation in collection of machines D
CN201410665567.0A2014-11-192014-11-19A kind of computational methods using the distributed data base load balancing for intersecting backupActiveCN104376087B (en)

Priority Applications (1)

Application NumberPriority DateFiling DateTitle
CN201410665567.0ACN104376087B (en)2014-11-192014-11-19A kind of computational methods using the distributed data base load balancing for intersecting backup

Applications Claiming Priority (1)

Application NumberPriority DateFiling DateTitle
CN201410665567.0ACN104376087B (en)2014-11-192014-11-19A kind of computational methods using the distributed data base load balancing for intersecting backup

Publications (2)

Publication NumberPublication Date
CN104376087Atrue CN104376087A (en)2015-02-25
CN104376087B CN104376087B (en)2017-09-29

Family

ID=52554994

Family Applications (1)

Application NumberTitlePriority DateFiling Date
CN201410665567.0AActiveCN104376087B (en)2014-11-192014-11-19A kind of computational methods using the distributed data base load balancing for intersecting backup

Country Status (1)

CountryLink
CN (1)CN104376087B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN105574127A (en)*2015-12-122016-05-11天津南大通用数据技术股份有限公司Quasi real-time disaster recovery method of distributed database system
CN105577756A (en)*2015-12-122016-05-11天津南大通用数据技术股份有限公司Distributed database log collection and load regulation system adopting cross backup and method thereof
CN106372161A (en)*2016-08-312017-02-01天津南大通用数据技术股份有限公司Distributive database data backup piece deployment method and device
WO2018028229A1 (en)*2016-08-102018-02-15华为技术有限公司Data shard storage method, device and system
CN108932104A (en)*2017-05-252018-12-04腾讯科技(深圳)有限公司A kind of data processing method, device and processing server
CN109408280A (en)*2017-08-172019-03-01北京金山云网络技术有限公司Data back up method, apparatus and system
CN110209679A (en)*2019-04-042019-09-06特斯联(北京)科技有限公司A kind of date storage method for promoting access efficiency, terminal device
CN110231977A (en)*2018-03-052019-09-13中兴通讯股份有限公司Processing method, device, storage medium and the electronic device of database
CN112241407A (en)*2020-09-112021-01-19重庆锐云科技有限公司Golf course member data processing method, customer management system and storage medium
CN115033390A (en)*2022-08-092022-09-09阿里巴巴(中国)有限公司Load balancing method and device
CN115994037A (en)*2023-03-232023-04-21天津南大通用数据技术股份有限公司Cluster database load balancing method and device
CN118573683A (en)*2024-07-312024-08-30广州智威智能科技有限公司Asynchronous balancing method and platform for intelligent campus service
CN120104344A (en)*2025-04-302025-06-06天津南大通用数据技术股份有限公司 A distributed database sharding computing system

Citations (5)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN101126992A (en)*2006-08-152008-02-20国际商业机器公司Method and system for dispensing multiple tasks at multiple node of network
US20120166492A1 (en)*2010-12-222012-06-28Microsoft CorporationDatabase transfers using constraint free data
US20140025704A1 (en)*2012-07-192014-01-23International Business Machines CorporationQuery method for a distributed database system and query apparatus
CN103810244A (en)*2013-12-092014-05-21北京理工大学Distributed data storage system expansion method based on data distribution
CN104144127A (en)*2013-05-082014-11-12华为软件技术有限公司 Load Balancing Methods and Load Balancers

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN101126992A (en)*2006-08-152008-02-20国际商业机器公司Method and system for dispensing multiple tasks at multiple node of network
US20120166492A1 (en)*2010-12-222012-06-28Microsoft CorporationDatabase transfers using constraint free data
US20140025704A1 (en)*2012-07-192014-01-23International Business Machines CorporationQuery method for a distributed database system and query apparatus
CN104144127A (en)*2013-05-082014-11-12华为软件技术有限公司 Load Balancing Methods and Load Balancers
CN103810244A (en)*2013-12-092014-05-21北京理工大学Distributed data storage system expansion method based on data distribution

Cited By (17)

* Cited by examiner, † Cited by third party
Publication numberPriority datePublication dateAssigneeTitle
CN105574127A (en)*2015-12-122016-05-11天津南大通用数据技术股份有限公司Quasi real-time disaster recovery method of distributed database system
CN105577756A (en)*2015-12-122016-05-11天津南大通用数据技术股份有限公司Distributed database log collection and load regulation system adopting cross backup and method thereof
WO2018028229A1 (en)*2016-08-102018-02-15华为技术有限公司Data shard storage method, device and system
US10942828B2 (en)2016-08-102021-03-09Huawei Technologies Co., Ltd.Method for storing data shards, apparatus, and system
CN106372161A (en)*2016-08-312017-02-01天津南大通用数据技术股份有限公司Distributive database data backup piece deployment method and device
CN108932104A (en)*2017-05-252018-12-04腾讯科技(深圳)有限公司A kind of data processing method, device and processing server
CN109408280A (en)*2017-08-172019-03-01北京金山云网络技术有限公司Data back up method, apparatus and system
CN110231977A (en)*2018-03-052019-09-13中兴通讯股份有限公司Processing method, device, storage medium and the electronic device of database
CN110209679B (en)*2019-04-042020-01-31特斯联(北京)科技有限公司data storage method and terminal equipment for improving access efficiency
CN110209679A (en)*2019-04-042019-09-06特斯联(北京)科技有限公司A kind of date storage method for promoting access efficiency, terminal device
CN112241407A (en)*2020-09-112021-01-19重庆锐云科技有限公司Golf course member data processing method, customer management system and storage medium
CN112241407B (en)*2020-09-112023-06-06重庆锐云科技有限公司Golf course member data processing method, client management system and storage medium
CN115033390A (en)*2022-08-092022-09-09阿里巴巴(中国)有限公司Load balancing method and device
CN115994037A (en)*2023-03-232023-04-21天津南大通用数据技术股份有限公司Cluster database load balancing method and device
CN118573683A (en)*2024-07-312024-08-30广州智威智能科技有限公司Asynchronous balancing method and platform for intelligent campus service
CN118573683B (en)*2024-07-312024-09-27广州智威智能科技有限公司Asynchronous balancing method and platform for intelligent campus service
CN120104344A (en)*2025-04-302025-06-06天津南大通用数据技术股份有限公司 A distributed database sharding computing system

Also Published As

Publication numberPublication date
CN104376087B (en)2017-09-29

Similar Documents

PublicationPublication DateTitle
CN104376087A (en)Load balance calculation method for distributed database adopting cross backups
EP3738045B1 (en)Distributed partitioned map reduce using a data fabric
EP3235227B1 (en)Replicated database distribution for workload balancing after cluster reconfiguration
US20170206140A1 (en)System and method for building a point-in-time snapshot of an eventually-consistent data store
US20120143873A1 (en)Method and apparatus for updating a partitioned index
US8713125B2 (en)Method and system for scaling usage of a social based application on an online social network
US11609914B2 (en)Policy-based storage structure distribution
CN101984632A (en)Load distributing method, device and server in distributed cache system
CN101534322A (en)File upload system and file upload method
US10019503B2 (en)Database transfers using constraint free data
CN106777311B (en)Flight space state caching method and system
US20160234129A1 (en)Communication system, queue management server, and communication method
CN104750757A (en)Data storage method and equipment based on HBase
EP2940587A1 (en)Computer, control device for computer system, and recording medium
JP5760350B2 (en) Information processing system
KR101704928B1 (en)System and method for distributively storing files using gpu
JP5691306B2 (en) Information processing system
KR101718397B1 (en)System and method for distributively storing files
US10171570B2 (en)Information processing apparatus
US20190158585A1 (en)Systems and Methods for Server Failover and Load Balancing
US8468181B2 (en)Distributed metadata cache
US20150088826A1 (en)Enhanced Performance for Data Duplication
CN109327520B (en)Method and device for establishing connection between terminal and server node
CN104572754B (en)A kind of Database Systems, Database Systems access method and device
JP5745445B2 (en) Management device and program

Legal Events

DateCodeTitleDescription
C06Publication
PB01Publication
C10Entry into substantive examination
SE01Entry into force of request for substantive examination
GR01Patent grant
GR01Patent grant

[8]ページ先頭

©2009-2025 Movatter.jp