Background technology
In the human society historical floods of 20,000 years, 5 scientific and technological revolutions are experienced by: the scientist being representative with Galileo, Copernius, newton etc., the first time scientific and technological revolution brought in the field such as astronomy, physics nearly; With the second time scientific and technological revolution that the invention of steam engine and application and machinery operation replace hand labour to be outstanding feature; With the third time scientific and technological revolution that power technology and internal combustion engine invention are outstanding feature; With the 4th scientific and technological revolution that evolutionism, the theory of relativity, quantum theory etc. are representative; There is the 5th scientific and technological revolution brought in electronics calculating, information network.After this, what large data, cloud computing, mobile Internet and Internet of Things etc. brought will be the 6th scientific and technological revolution.
The appearance of cloud computing is the inevitable outcome that Information Technology Development and information-intensive society demand arrive certain phase.On the one hand, scientific and technical basis has been established by the human society cloud epoch society that develops into of Intel Virtualization Technology, distributed computing technology, broadband internet technology, SOA technology and automatic management technology; In addition on the one hand, no matter when, where, who, what thing, human society expect realize interconnect, knowledge sharing, collaborative work new demand, accelerate the process of information-intensive society, in this process, to Pu Hui, reliable, low cost, dynamical technological means and implementation pattern in the urgent need to, expedited the emergence of cloud computing
Cloud computing, from technology and business two aspects, is overturned traditional application model.Changed as required by resource centralization, service network, charging and demand elasticity etc., solve that asset utilization ratio under existing information technological accumulation and inheritance is lower, resource requirement dispersion, system redundancy, be difficult to management and the problem such as procurement cycle is long, while reduction IT service provides cost, effectively improve the flexibility that service provides, for informationalized development is provided convenience.
That both at home and abroad had some enterprises to provide to be a splendid legacy, To enterprises and individual cloud computing service, change the custom of consumer's work, life, study, amusement, as To enterprises client cloud data center, cloud office, cloud monitoring, and store towards the cloud of personal user, the reading of cloud video, cloud, cloud music etc., possess shared, synchronous, towards features such as movement, Dynamic Acquisition, multiple terminals application.These successful services obtain the accreditation of user
Phicloud cloud platform is taken advantage of a situation according to the call of the market demand and national cloud computing production domesticization and give birth to, take OpenStack as support, using its resource management rear end as cloud platform.OpenStack is a cloud platform of increasing income, and is developed cooperatively by NASA and Rackspace, and it to be increased income realization to the one of AWS (AmazonWebService, Amazon cloud computing service platform).OpenStack provides the platform that is disposed cloud, for publicly-owned cloud, privately owned cloud provide easily extensible, flexibly cloud computing platform.The development of OpenStack is very fast, but OpenStack merely provides the distribution of resource, the acquisition function that scheduling and part resource use, and does not provide the important component-Billing component of publicly-owned cloud platform.This just needs the Billing component of all publicly-owned cloud manufacturer exploitations oneself, and the charging of PhiCloud cloud is exactly produce under this background.The vertical pattern that the framework of the Billing component of publicly-owned cloud manufacturer domestic is at present main or traditional, the autgmentability of this pattern is very poor, needs for billing function provides special hardware, software, the support of IT manpower, this materially increases the burden of enterprise and the waste of resource.
Summary of the invention
The shortcoming of prior art in view of the above, the object of the present invention is to provide a kind of PhiCloud cloud charging method and system, be vertical pattern for solving the architecture mode of existing cloud Billing component, autgmentability is poor, needs the problem that special software and hardware and human resources are supported.
For achieving the above object and other relevant objects, the invention provides a kind of PhiCloud cloud charge system, described PhiCloud cloud charge system comprises: data acquisition unit cluster, comprises at least 1 data acquisition unit, gathers original charging data; Data-analyzing machine cluster, is connected with described data acquisition unit cluster, comprises at least 1 data-analyzing machine, adopts the described original charging data of MapReduce process, obtains charging result data; Accounting server cluster, is connected with described data-analyzing machine cluster, comprises at least 1 accounting server, is bill according to predetermined charging policy by described charging result data transformations; Resource Scheduler, be connected respectively with accounting server cluster with described data acquisition unit cluster, data-analyzing machine cluster, according to the resource load of predetermined scheduling strategy integrated management schedule said data collector cluster, data-analyzing machine cluster and accounting server cluster.
Alternatively, described Resource Scheduler comprises: Registering modules, receives and preserves the cluster information of self of the component register needing scheduling of resource; The described assembly of scheduling of resource that needs comprises described data acquisition unit cluster, data-analyzing machine cluster or/and accounting server cluster; Data acquisition module, is connected with described Registering modules, gathers the load information of each node in cluster according to the cluster information of described component register; Resource dispatching strategy administration module, is provided with the threshold value that resource needs to reschedule; Scheduling of resource module, is connected with data acquisition module respectively with described resource dispatching strategy administration module, and the actual loading information according to described threshold value and described assembly carries out scheduling of resource to described assembly.
Alternatively, described data acquisition unit uses the Libvirt storehouse of increasing income to gather the various state informations operating in the virtual machine on host in OpenStack, using described various state information as original charging data stored in local billing document or distributed file system; Described original charging data is published on described data-analyzing machine by issuing passage by described data acquisition unit after the upload command receiving described data-analyzing machine; Described data acquisition unit is the ceilometer assembly of OpenStack.
Alternatively, described data-analyzing machine comprises: distributed file system, stores the original charging data that described data acquisition unit is uploaded; Extraction module, is connected with described distributed file system, extracts ID and the timestamp of virtual machine from described original charging data; The time that described timestamp produces for identifying described original charging data; Map function module, is connected with described extraction module, using described ID and timestamp corresponding input parameter key and value as Map function respectively, is sorted out by the timestamp of the virtual machine of correspondence, generate the timestamp list corresponding with described ID according to described ID; Reduce function module, is connected with described Map function module, is processed by the input parameter of described timestamp list as Reduce function, obtains the charging result data of the virtual machine corresponding with described ID.
Alternatively, described accounting server comprises: charging policy administration module, manages charging policy; Charge information generation module, be connected with described charging policy administration module, access the charging policy that described charging policy administration module obtains current host user selection, use the charging result data of the virtual machine that current host described in selected charging policy process runs, generate the bill of active user; Billing database, is connected with described charge information generation module, stores described bill; Charging display module, is connected with described billing database, shows described bill by the form of webpage.
The present invention also provides a kind of PhiCloud cloud charging method, and described PhiCloud cloud charging method comprises the following steps: utilize the data acquisition unit cluster comprising at least 1 data acquisition unit to gather original charging data; Utilize the data-analyzing machine cluster comprising at least 1 data-analyzing machine to adopt the described original charging data of MapReduce process, obtain charging result data; The accounting server cluster comprising at least 1 accounting server is utilized to be bill according to predetermined charging policy by described charging result data transformations; Utilize Resource Scheduler according to the resource load of predetermined scheduling strategy integrated management schedule said data collector cluster, data-analyzing machine cluster and accounting server cluster.
Alternatively, described PhiCloud cloud charging method also comprises: receive and preserve the cluster information of self of the component register needing scheduling of resource; The described assembly of scheduling of resource that needs comprises described data acquisition unit cluster, data-analyzing machine cluster or/and accounting server cluster; The load information of each node in cluster is gathered according to the cluster information of described component register; The threshold value that resource needs to reschedule is set; Actual loading information according to described threshold value and described assembly carries out scheduling of resource to described assembly.
Alternatively, described PhiCloud cloud charging method also comprises: described data acquisition unit uses the Libvirt storehouse of increasing income to gather the various state informations operating in the virtual machine on host in OpenStack, using described various state information as original charging data stored in local billing document or distributed file system; Described original charging data is published on described data-analyzing machine by issuing passage by described data acquisition unit after the upload command receiving described data-analyzing machine; Described data acquisition unit is the ceilometer assembly of OpenStack.
Alternatively, described PhiCloud cloud charging method also comprises: the original charging data utilizing distributed file system to store described data acquisition unit to upload; ID and the timestamp of virtual machine is extracted from described original charging data; The time that described timestamp produces for identifying described original charging data; Using described ID and timestamp corresponding input parameter key and value as Map function respectively, according to described ID, the timestamp of the virtual machine of correspondence is sorted out, generate the timestamp list corresponding with described ID; The input parameter of described timestamp list as Reduce function is processed, obtains the charging result data of the virtual machine corresponding with described ID.
Alternatively, described PhiCloud cloud charging method also comprises: manage charging policy; Access obtains the charging policy that current host user selects, and uses the charging result data of the virtual machine that current host described in selected charging policy process runs, and generates the bill of active user; Store described bill; Described bill is shown by the form of webpage.
As mentioned above, PhiCloud cloud charging method of the present invention and system, have following beneficial effect:
The present invention is that Phicloud cloud platform provides billing solution, for the billing of services of cloud platform provides support, complement each other with cloud platform, can realize on existing software and hardware basis, without the need to special software and hardware and human resources support, autgmentability is better, realize more efficient, more saving resource, is more convenient for safeguarding, is easier to the billing function expanded.
Embodiment
Below by way of specific instantiation, embodiments of the present invention are described, those skilled in the art the content disclosed by this specification can understand other advantages of the present invention and effect easily.The present invention can also be implemented or be applied by embodiments different in addition, and the every details in this specification also can based on different viewpoints and application, carries out various modification or change not deviating under spirit of the present invention.It should be noted that, when not conflicting, the feature in following examples and embodiment can combine mutually.
It should be noted that, the diagram provided in following examples only illustrates basic conception of the present invention in a schematic way, then only the assembly relevant with the present invention is shown in graphic but not component count, shape and size when implementing according to reality is drawn, it is actual when implementing, and the kenel of each assembly, quantity and ratio can be a kind of change arbitrarily, and its assembly layout kenel also may be more complicated.
Refer to Fig. 1, the invention provides a kind of PhiCloud cloud charge system, described PhiCloud cloud charge system 100 comprises: data acquisition unit cluster, data-analyzing machine cluster, accounting server cluster, Resource Scheduler.
Described data acquisition unit cluster comprises at least 1 data acquisition unit 110, gathers original charging data.
Further, the general principle of described data acquisition unit uses the Libvirt storehouse of increasing income to gather the various state informations operating in the virtual machine on host in OpenStack, using described various state information as original charging data stored in local billing document; Described data acquisition unit after the upload command receiving described data-analyzing machine by issuing in distributed file system that described original charging data to be published in data-analyzing machine by passage so that described data-analyzing machine is transferred at any time.
Described data acquisition unit can be preferably the ceilometer assembly of OpenStack.Because the ceilometer assembly of Openstack has achieved the function of data acquisition, so do not need to repeat invention wheel again, ceilometer is directly utilized to realize this function, adopt the mode of polling agent (PollingAgent), call the calculating of OpenStack, network, mirror image, the interface image data that the assemblies such as object storage provide, the data of collection are published to data-analyzing machine by issuing passage, and specific implementation structure is shown in Figure 2.In OpenStack environment, virtual machine may be in different states, after therefore data acquisition unit starts, needs the state monitoring the virtual machine that host runs on backstage.If virtual machine is in active state, data acquisition unit will call LibvirtAPI periodically obtain virtual machine active state under various original charging datas.Described original charging data comprise UUID, the VCPU of virtual machine, internal memory, the read-write speed of disk, the size of disk, network interface card upload downloading rate and uploading of network downloads total amount etc.The data obtained become original charging data, stored in local billing document after process.When after the upload command that data acquisition unit receives from data-analyzing machine, the original charging data in local billing document will be read, then upload in data-analyzing machine.When virtual machine is in inactive state, data acquisition unit also can evaluate and test out its state, at this moment can suspend the original charging data collection to virtual machine, but still can continue to carry out periodic poll access to virtual machine, to ensure the original charging data can not losing virtual machine to be measured.When virtual machine changes state, data acquisition unit can determine its state run, and starts the preprocessed original state information gathering this virtual machine immediately, and its concrete collecting flowchart is shown in Figure 3.
Described data-analyzing machine cluster is connected with described data acquisition unit cluster, comprises at least 1 data-analyzing machine 120, adopts the described original charging data of MapReduce process, obtains charging result data.
Further, shown in Figure 4, described data-analyzing machine 120 comprises: distributed file system 121, extraction module 122, Map function module 123, Reduce function module 124.
Described distributed file system 121 stores the original charging data that described data acquisition unit 110 is uploaded.Distributed file system (DistributedFileSystem) refers to that the physical memory resources of file system management is not necessarily connected directly between on local node, but is connected with node by computer network.The design of distributed file system is based on Client/Server pattern.A typical network may comprise multiple server for multi-user access.In addition, ad-hoc nature allows some systems to play the part of the dual role of client-server.Such as, user " can deliver " one and allow the catalogue of other client access, once accessed, this catalogue concerning client computer as the use local drive.
Described extraction module 122 is connected with described distributed file system 121, extracts ID and the timestamp of virtual machine from described original charging data; The time that described timestamp produces for identifying described original charging data.
Described Map function module 123 is connected with described extraction module 122, using described ID and timestamp corresponding input parameter key and value as Map function respectively, according to described ID, the timestamp of the virtual machine of correspondence is sorted out, generate the timestamp list corresponding with described ID.
Described Reduce function module 124 is connected with described Map function module 123, is processed by the input parameter of described timestamp list as Reduce function, obtains the charging result data of the virtual machine corresponding with described ID.
Described data-analyzing machine is responsible for the data of data acquisition unit collection to arrange, first it is required to determine that it charging initial time and charging end time.Charging initial time refers to the time that user starts virtual machine first time.Because user once may apply for multiple stage virtual machine, only have when user deletes all virtual machines, charging could stop.Determine charging initial time and after the end time, just need to process original charging data, just can obtain the service time of every platform virtual machine.The present invention adopts MapReduce process original charging data.Wherein, MapReduce is a kind of programming model, for the concurrent operation of large-scale dataset (being greater than 1TB).Concept " Map (mapping) " and " Reduce (reduction) ", and their main thought, all borrow from Functional Programming, the characteristic of borrowing from vector programming language in addition.It is very easy to programming personnel when can not distributed parallel programming, the program of oneself is operated in distributed system.Current software simulating is appointment Map (mapping) function, one group of key-value pair is used for be mapped to one group of new key-value pair, specify concurrent Reduce (reduction) function, each being used for ensureing in the key-value pair of all mappings shares identical key group.The basic handling principle of employing MapReduce process original charging data is: from original charging data, first extract virtual machine ID and timestamp.The ID of virtual machine is used for unique identification virtual machine, and the time that time stamp T imeStamp produces for identifying this original charging data.Using the ID of virtual machine and timestamp as input parameter key and value of Map function.The major function of Map function is sorted out by the timestamp of virtual machine according to the ID of virtual machine, and the intermediate object program of generation is the corresponding timestamp list of a virtual machine instance ID.Then using the input parameter of intermediate object program as Reduce function.Reduce function carrys out processing time stamp list according to virtual machine instance ID, and the difference value of after list being deducted to the value of last item judges, when the differences of two are less than certain threshold time, can carry out accumulation process to this value, idiographic flow as shown in Figure 5.
Described accounting server cluster is connected with described data-analyzing machine cluster, comprises at least 1 accounting server 130, is bill according to predetermined charging policy by described charging result data transformations.
Further, shown in Figure 6, described accounting server 130 comprises: charging policy administration module 131, charge information generation module 132, billing database 133, charging display module 134.
Described charging policy administration module 131 pairs of charging policy manage, and comprise the existing charging policy of inquiry system, increase charging policy, amendment or the original charging policy of deletion system etc.
Described charge information generation module 132 is connected with described charging policy administration module 131, access the charging policy that described charging policy administration module obtains current host user selection, the charging result data of the virtual machine that the described current host using selected charging policy process to be stored in distributed file system runs, generate the bill of active user.That is, use the information relevant to charging stored in distributed file system described in this charging policy process, obtain the cost information of virtual machine, and by cost information stored in billing database, for charging component exhibiting.
Described billing database 133 is connected with described charge information generation module 132, stores described bill.
Described charging display module 134 is connected with described billing database 133, shows described bill by the form of webpage.Described charging display module 134 reads the cost information (i.e. bill) of the virtual machine stored in billing database 133, then the form of these cost informations by webpage is displayed.
The workflow of described accounting server 130 is shown in Figure 7, its charging result data produced according to data-analyzing machine 120, and according to the trigger condition that charging policy sets, selects corresponding charging policy to calculate the cost information of host.Account module can go user account to take off expense according to cost information, and the middle financial information produced can be put in storage in order to financial audit later.
Described Resource Scheduler 140 is connected with accounting server cluster respectively with described data acquisition unit cluster, data-analyzing machine cluster, according to the resource load of predetermined scheduling strategy integrated management schedule said data collector cluster, data-analyzing machine cluster and accounting server cluster.Namely, described Resource Scheduler 140 judges the resource load situation of described data acquisition unit cluster, data-analyzing machine cluster and accounting server cluster, Dynamic expansion or the resource reduced in the cluster resource pond of described data acquisition unit, data-analyzing machine and accounting server.
Further, shown in Figure 8, described Resource Scheduler 140 comprises: Registering modules 141, data acquisition module 142, resource dispatching strategy administration module 143, scheduling of resource module 144.
Described Registering modules 141 receives and preserves the cluster information of self of the component register needing scheduling of resource; The described assembly of scheduling of resource that needs comprises described data acquisition unit cluster, data-analyzing machine cluster or/and accounting server cluster; Described Registering modules can preserve the metadata of these assemblies, if the cluster information of assembly changes, assembly also will notify that Registering modules upgrades log-on message in time.
Described data acquisition module 142 is connected with described Registering modules 141, gathers the load information of each node in cluster according to the cluster information of described component register.Described load information comprises cpu load, memory usage, disk I/O, and network interface card imports and exports the information such as flow.
Described resource dispatching strategy administration module 143 is provided with the threshold value that resource needs to reschedule.Such as, described threshold value be cluster average CPU load reach within the scope of certain hour how many times need for cluster increase or reduce resource; Described threshold value comprises limits value or the value range of polytype (as cpu load, memory usage, disk I/O, network interface card import and export flow etc.).
Described scheduling of resource module 144 is connected with data acquisition module 142 respectively with described resource dispatching strategy administration module 143, and the actual loading information according to described threshold value and described assembly carries out scheduling of resource to described assembly.Concrete scheduling of resource content comprises to needing add in the assembly of scheduling of resource or remove calculating, or storage resources.The workflow of described Resource Scheduler 140 is shown in Figure 9.
Data-analyzing machine of the present invention adopts HDFS distributed file system to store billing document, MapReduce algorithm is adopted to process original charging data, there is better autgmentability and better efficiency, when the load of data-analyzing machine cluster is higher, Resource Scheduler can dynamically for data-analyzing machine increases computational resource or storage resources, to ensure the efficient of data handling procedure.When the load of data-analyzing machine cluster is lower, Resource Scheduler dynamically can reduce the resource of data-analyzing machine cluster, can reduce the wasting of resources like this, improve the resource utilization of whole platform.
The resource of the whole charging platform of Resource Scheduler integrated management of the present invention, reasonably increases or reduces the core work that resource is Resource Scheduler.Resource Scheduler is after scheduling strategy sets, and following resource traffic control, without the need to manual intervention, all completes automatically, greatly reduces the input of manpower, makes people can drop into the more time to more creative work.In addition, Resource Scheduler also can gather the resource service condition of each cluster assembly, early warning can be sent when resource occurs abnormal, although Resource Scheduler can Automatic dispatching resource, but when needs manual intervention, can formulate according to these monitor messages and intervene prediction scheme accordingly, whole system can be made like this to have higher availability.
The present invention also provides a kind of PhiCloud cloud charging method; the method can be realized by PhiCloud cloud charge system of the present invention; but the implement device of the method includes but not limited to the structure of the PhiCloud cloud charge system that the present invention enumerates; the technical scheme that solution the present invention that the amendment of the prior art that every principle according to the present invention is made, expansion and replacement realize deals with problems, is all included in protection scope of the present invention.
Shown in Figure 10, described PhiCloud cloud charging method comprises:
S1, utilizes the data acquisition unit cluster comprising at least 1 data acquisition unit to gather original charging data.Described original charging data comprise UUID, the VCPU of virtual machine, internal memory, the read-write speed of disk, the size of disk, network interface card upload downloading rate and uploading of network downloads total amount etc.
Further, a kind of implementation procedure of described step S1 comprises: described data acquisition unit uses the Libvirt storehouse of increasing income to gather the various state informations operating in the virtual machine on host in OpenStack, using described various state information as original charging data stored in local billing document or distributed file system; Described original charging data is published on described data-analyzing machine by issuing passage by described data acquisition unit after the upload command receiving described data-analyzing machine; Described data acquisition unit is the ceilometer assembly of OpenStack.
S2, utilizes the data-analyzing machine cluster comprising at least 1 data-analyzing machine to adopt the described original charging data of MapReduce process, obtains charging result data.
Further, shown in Figure 11, a kind of implementation procedure of described step S2 comprises:
S21, the original charging data utilizing distributed file system to store described data acquisition unit to upload.
S22, extracts ID and the timestamp of virtual machine from described original charging data; The time that described timestamp produces for identifying described original charging data;
S23, using described ID and timestamp corresponding input parameter key and value as Map function respectively, sorts out the timestamp of the virtual machine of correspondence according to described ID, generates the timestamp list corresponding with described ID;
S24, processes the input parameter of described timestamp list as Reduce function, obtains the charging result data of the virtual machine corresponding with described ID.
S3, utilizes the accounting server cluster comprising at least 1 accounting server to be bill according to predetermined charging policy by described charging result data transformations.
Further, shown in Figure 12, a kind of implementation procedure of described step S3 comprises:
S31, manages charging policy, comprises the existing charging policy of inquiry system, increases charging policy, amendment or the original charging policy of deletion system etc.
S32, access obtains the charging policy that current host user selects, and uses the charging result data of the virtual machine that current host described in selected charging policy process runs, and generates the bill of active user.That is, use the information relevant to charging stored in distributed file system described in this charging policy process, obtain the cost information of virtual machine, and by cost information stored in billing database, for charging component exhibiting.
S33, stores described bill.
S34, shows described bill by the form of webpage.
S4, utilizes Resource Scheduler according to the resource load of predetermined scheduling strategy integrated management schedule said data collector cluster, data-analyzing machine cluster and accounting server cluster.
Further, shown in Figure 13, a kind of implementation procedure of described step S4 comprises:
S41, receives and preserves the cluster information of self of the component register needing scheduling of resource; The described assembly of scheduling of resource that needs comprises described data acquisition unit cluster, data-analyzing machine cluster or/and accounting server cluster; Described Registering modules can preserve the metadata of these assemblies, if the cluster information of assembly changes, assembly also will notify that Registering modules upgrades log-on message in time.
S42, gathers the load information of each node in cluster according to the cluster information of described component register; Described load information comprises cpu load, memory usage, disk I/O, and network interface card imports and exports the information such as flow.
S43, arranges the threshold value that resource needs to reschedule; Such as, described threshold value be cluster average CPU load reach within the scope of certain hour how many times need for cluster increase or reduce resource; Described threshold value comprises limits value or the value range of polytype (as cpu load, memory usage, disk I/O, network interface card import and export flow etc.).
S44, the actual loading information according to described threshold value and described assembly carries out scheduling of resource to described assembly.Concrete scheduling of resource content comprises to needing add in the assembly of scheduling of resource or remove calculating, or storage resources.
The protection range of PhiCloud cloud charging method of the present invention is not limited to the step that the present embodiment enumerates and performs sequencing; the schemes such as every principle according to the present invention adjusts the sequence of steps of the prior art at the present embodiment place, step replacement, step increase and decrease; only can solve problem of the present invention, all be included in protection scope of the present invention.
Phicloud cloud charging method of the present invention and system are the billing solutions of Phicloud cloud platform, its billing of services being cloud platform provides support, cloud charging method of the present invention and system and cloud platform complement each other, resource that cloud charging is cloud platform expenditure provides metering and evaluates, and cloud platform provides necessary calculating, storage, Internet resources for cloud charging.
The present invention is that Phicloud cloud platform provides billing solution, for the billing of services of cloud platform provides support, complement each other with cloud platform, can realize on existing software and hardware basis, without the need to special software and hardware and human resources support, autgmentability is better, realize more efficient, more saving resource, is more convenient for safeguarding, is easier to the billing function expanded.
In sum, the present invention effectively overcomes various shortcoming of the prior art and tool high industrial utilization.
Above-described embodiment is illustrative principle of the present invention and effect thereof only, but not for limiting the present invention.Any person skilled in the art scholar all without prejudice under spirit of the present invention and category, can modify above-described embodiment or changes.Therefore, such as have in art usually know the knowledgeable do not depart from complete under disclosed spirit and technological thought all equivalence modify or change, must be contained by claim of the present invention.