CN113411364B - Resource acquisition method and device and server - Google Patents

Abstract

The invention provides a resource acquisition method, a resource acquisition device and a server, which are applied to node equipment of a CDN (content delivery network); according to a resource access request from a client, determining address information of a resource to be accessed and a fragment index number of first fragment data in the resource to be accessed; acquiring first fragment data in the resource to be accessed according to the address information and the fragment index number; determining version information of the resource to be accessed through the first fragment data; and acquiring the fragment data except the first fragment data in the resource to be accessed according to the address information, the fragment index number of the fragment data except the first fragment data in the resource to be accessed and the version information. According to the method and the device, the version information of the resource is determined through the first fragment data, and the fragment data except the first fragment data is obtained based on the version information, so that the consistency of the versions of the fragment data of the resource to be accessed is ensured, the accuracy and the stability of resource obtaining are improved, and the user experience is improved.

Description

Resource acquisition method and device and server

Technical Field

The invention relates to the technical field of internet, in particular to a resource acquisition method, a resource acquisition device and a server.

Background

In a node of a CDN network, a cache device usually manages a plurality of disks; and the cache device performs bare disk management on a plurality of disks through cache software and directly performs add-delete read-write on data in the disks. Typically, the cache software sets up one management thread for each disk. The multiple fragments of one resource are usually stored on different disks, if the resource needs to be read, the management threads of the multiple disks which need to store the resource fragment all execute the task of reading the fragment; if the version of the resource is updated in the reading process, the management threads of the multiple disks can easily read resource fragments of different versions, so that the resource fragments of different versions are spliced together and then returned to the client, and the versions of the resource fragments are inconsistent, which easily causes the reading error of the resource of the client.

Disclosure of Invention

In view of this, an object of the present invention is to provide a resource obtaining method, device and server, so as to ensure consistency of versions between fragmented data of resources to be accessed, and improve accuracy and stability of resource obtaining, thereby improving user experience.

In a first aspect, an embodiment of the present invention provides a resource obtaining method, which is applied to a node device of a CDN network; the method comprises the following steps: according to a resource access request from a client, determining address information of a resource to be accessed and a fragment index number of first fragment data in the resource to be accessed; acquiring first fragment data in the resource to be accessed according to the address information and the fragment index number of the first fragment data; determining version information of the resource to be accessed through the first fragment data; acquiring the fragmented data except the first fragmented data in the resource to be accessed according to the address information, the fragmented index number of the fragmented data except the first fragmented data in the resource to be accessed and the version information of the resource to be accessed, wherein the resource version indicated by the version information of the fragmented data except the first fragmented data in the resource to be accessed is the same as the resource version indicated by the version information of the first fragmented data; and determining the first fragment data and the fragment data except the first fragment data as the resource to be accessed corresponding to the acquired resource access request.

With reference to the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where the step of determining, according to a resource access request from a client, address information of a resource to be accessed and a segment index number of first segment data in the resource to be accessed includes: extracting address information of the resource to be accessed and a resource range of the resource to be accessed indicated by the resource access request from a client; and acquiring the fragment index number of the first fragment data from the resource range.

With reference to the first aspect, an embodiment of the present invention provides a second possible implementation manner of the first aspect, where the node device includes multiple disks; the disk is used for storing the fragment data; the step of obtaining the first fragment data in the resource to be accessed according to the address information and the fragment index number of the first fragment data includes: calculating to obtain a keyword of the first fragment data according to the address information, the fragment index number of the first fragment data in the resource to be accessed and preset default version information; calculating by adopting a consistent hash algorithm according to the keyword of the first fragmented data to obtain a first target disk where the first fragmented data is located; and acquiring the first fragment data in the resource to be accessed through the first target disk.

With reference to the second possible implementation manner of the first aspect, an embodiment of the present invention provides a third possible implementation manner of the first aspect, where each disk corresponds to one thread, and the thread is configured to manage fragment data in the disk corresponding to the thread; the step of obtaining the first fragment data in the resource to be accessed through the first target disk includes: inquiring whether the first fragment data in the resource to be accessed exists in the first target disk through a thread corresponding to the first target disk; under the condition that first fragmented data in resources to be accessed exist in a first target disk, reading the first fragmented data from the first target disk; and under the condition that the first fragment data in the resource to be accessed does not exist in the first target disk, acquiring the first fragment data from the superior equipment of the node equipment.

With reference to the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where the step of determining, through the first fragment data, version information of a resource to be accessed includes: under the condition that the first fragment data is acquired from the node equipment, reading the version information of the resource to be accessed from the response head of the first fragment data; under the condition that the first fragment data is acquired from the superior device of the node device, preset initial version information is set for the resource to be accessed corresponding to the first fragment data, and the initial version information is used as the version information of the resource to be accessed and is added to a response header of the first fragment data.

With reference to the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, where the step of obtaining the fragmented data in the resource to be accessed, except for the first fragmented data, according to the address information, the fragmented index number of the fragmented data in the resource to be accessed, except for the first fragmented data, and the version information of the resource to be accessed includes: extracting a resource range of a resource to be accessed from the resource access request; acquiring the fragment index numbers of the fragment data except the first fragment data from the resource range according to the data volume of the preset unit fragment data; aiming at each fragment data except the first fragment data in the resource to be accessed, calculating to obtain a keyword of the current fragment data according to the address information, the fragment index number of the current fragment data and the version information of the resource to be accessed; calculating by adopting a consistent hash algorithm according to the keywords of the current fragmented data to obtain a second target disk where the current fragmented data is located; and acquiring the current fragment data through the second target disk.

With reference to the fifth possible implementation manner of the first aspect, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, where the step of obtaining, by using the second target disk, current fragmented data includes: inquiring whether current fragment data exists in the second target disk through a thread corresponding to the second target disk; under the condition that the current fragment data exists in the second target disk, reading the current fragment data from the second target disk; and under the condition that the current fragment data does not exist in the second target disk, acquiring the current fragment data from the superior device of the node device.

With reference to the sixth possible implementation manner of the first aspect, an embodiment of the present invention provides a seventh possible implementation manner of the first aspect, where after the step of obtaining the current fragment data from the upper device of the node device, the method further includes: extracting a first data size of a resource to be accessed from the current fragment data; comparing whether the first data volume is the same as a second data volume of the resource to be accessed carried in the first fragment data of the resource to be accessed; if the current fragment data is the same as the first fragment data of the resource to be accessed, determining that the version of the current fragment data is the same as that of the first fragment data of the resource to be accessed, and calculating to obtain a keyword of the current fragment data according to the address information, the fragment index number of the current fragment data and the version information of the resource to be accessed; according to the keywords of the current fragmented data, a disk for storing the current fragmented data is obtained by adopting a consistent Hash algorithm; and storing the current fragment data to the calculated disk.

With reference to the seventh possible implementation manner of the first aspect, an embodiment of the present invention provides an eighth possible implementation manner of the first aspect, where the method further includes: if the first data volume is different from the second data volume, determining that the versions of the current fragment data and the first fragment data of the resource to be accessed are different, and deleting the first fragment data of the resource to be accessed stored in the node equipment; acquiring the first fragment data of the resource to be accessed from the superior device of the node device again, adding the updated version information of the resource to be accessed to the response head of the first fragment data, and storing the first fragment data; acquiring and storing the fragment data of the resource to be accessed except the first fragment data from the superior device of the node device; and acquiring and storing the fragment data of the resource to be accessed except the first fragment data from the superior equipment of the node equipment.

In a second aspect, an embodiment of the present invention further provides a resource obtaining apparatus, which is disposed in a node device of a CDN network; the device includes: the fragment index number determining module is used for determining the address information of the resource to be accessed and the fragment index number of the first fragment data in the resource to be accessed according to the resource access request from the client; the first fragment data acquisition module is used for acquiring the first fragment data in the resource to be accessed according to the address information and the fragment index number of the first fragment data; the version information determining module is used for determining the version information of the resource to be accessed through the first fragment data; the second fragment data acquisition module is used for acquiring the fragment data except the first fragment data in the resource to be accessed according to the address information, the fragment index number of the fragment data except the first fragment data in the resource to be accessed and the version information of the resource to be accessed, wherein the resource version indicated by the version information of the fragment data except the first fragment data in the resource to be accessed is the same as the resource version indicated by the version information of the first fragment data; and the to-be-accessed resource determining module is used for determining the first fragment data and the fragment data except the first fragment data as the to-be-accessed resource corresponding to the acquired resource access request.

With reference to the second aspect, an embodiment of the present invention provides a first possible implementation manner of the second aspect, where the segment index number determining module is further configured to: extracting address information of the resource to be accessed and a resource range of the resource to be accessed indicated by the resource access request from a client; and acquiring the fragment index number of the first fragment data from the resource range.

With reference to the second aspect, an embodiment of the present invention provides a second possible implementation manner of the second aspect, where the node device includes multiple disks; the disk is used for storing the fragment data; the first sliced data obtaining module is further configured to: calculating to obtain a keyword of the first fragment data according to the address information, the fragment index number of the first fragment data in the resource to be accessed and preset default version information; calculating by adopting a consistent hash algorithm according to the keyword of the first fragmented data to obtain a first target disk where the first fragmented data is located; and acquiring the first fragment data in the resource to be accessed through the first target disk.

In a third aspect, an embodiment of the present invention further provides a server, including a processor and a memory, where the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the resource obtaining method.

In a fourth aspect, embodiments of the present invention also provide a machine-readable storage medium storing machine-executable instructions, which when called and executed by a processor, cause the processor to implement the above-mentioned resource acquisition method.

The embodiment of the invention has the following beneficial effects:

the resource obtaining method, the resource obtaining device and the server are applied to node equipment of a CDN (content delivery network); firstly, according to a resource access request from a client, determining address information of a resource to be accessed and a fragment index number of first fragment data in the resource to be accessed; after first fragment data in the resource to be accessed is obtained according to the address information and the fragment index number of the first fragment data, determining version information of the resource to be accessed through the first fragment data; and then, according to the address information, the fragment index number of the fragment data except the first fragment data in the resource to be accessed and the version information of the resource to be accessed, obtaining the fragment data except the first fragment data in the resource to be accessed. According to the method, the version information of the resource is determined through the first fragment data, and the fragment data except the first fragment data is obtained based on the version information, so that the consistency of the versions of the fragment data of the resource to be accessed is ensured, the accuracy and the stability of resource obtaining are improved, and the user experience is improved.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention as set forth above.

In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a schematic diagram of an application scenario of a resource obtaining method according to an embodiment of the present invention;

fig. 2 is a flowchart of a resource obtaining method according to an embodiment of the present invention;

fig. 3 is a flowchart of another resource obtaining method according to an embodiment of the present invention;

fig. 4 is a flowchart of another resource acquisition method according to an embodiment of the present invention;

fig. 5 is a flowchart of a process of saving fragmented data obtained from a higher-level device of a node device in another resource obtaining method according to an embodiment of the present invention;

fig. 6 is an architecture diagram of a node device according to an embodiment of the present invention;

FIG. 7 is an architecture diagram of a cache software according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a resource acquisition apparatus according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of a server according to an embodiment of the present invention.

Detailed Description

The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

For convenience of understanding, an application scenario of the resource obtaining method is shown first, and as shown in fig. 1, the scenario includes a user terminal, a domain name system, a CDN (Content Delivery Network) node device, and a source station server.

The user terminal can be a mobile terminal (such as a tablet computer, a mobile phone and the like), a computer or special terminal equipment, and a client or a browser is installed on the user terminal; the user can send a resource access request through the user terminal, wherein the resource access request comprises website information of the resource to be accessed. The resource access request is first sent to the domain name system.

The domain name system generally includes a local recursive domain name server, a root domain name server, a top level domain name server, various levels of domain name servers, and the like. After receiving a resource access request sent by a user terminal, a Domain Name System obtains an IP address of a web server of a target web page in a common DNS (Domain Name System) resolution manner. In short, the local domain name server performs layer-by-layer analysis on the root domain name server, the top level domain name server corresponding to the website information of the resource to be accessed and each level of domain name servers to obtain an access address of a certain server capable of acquiring the resource to be accessed; and sends the access address of the server to the user terminal. After receiving an access address of a server which can acquire resources to be accessed, a user terminal sends a resource access request to the server.

The server capable of acquiring the resource to be accessed may be an edge node in a node device of the CDN network. The node equipment of the CDN network comprises an edge node, a middle layer node and an upper layer node. The number of the edge nodes is the largest, and the edge nodes directly serve network users (also called netizens); the middle layer node is mainly used for converging the back-source request of the edge node, does not directly serve the network user, and has fewer number; the upper node mainly converges the request of the middle node back to the source, does not directly serve the network user, and has the least number; typically, a typical network user does not pay for bandwidth generated by middle or upper level nodes.

Node equipment and a source station of the CDN belong to a content delivery network; the content distribution network is an intelligent virtual network built on the basis of the existing network; website resources are saved in a source station of the content distribution network, and the website resources in the source station are distributed to edge nodes through an upper layer node and a middle layer node; the edge node is a node device of the CDN network for the user terminal to directly access and obtain the resources to be accessed. When a resource access request is received, the edge node does not have a resource to be accessed corresponding to the resource access request, and the edge node can acquire the resource to be accessed from the middle node; if the middle layer node does not have the resource to be accessed, the resource to be accessed can be obtained from the upper layer node; if the upper node does not have the resource to be accessed, the resource to be accessed can be obtained from the source station; the above process of acquiring the resource to be accessed by the node device at the upper layer may be referred to as a back source pull (i.e. the above back source request).

Cache devices are usually arranged in node devices of each CDN network. One cache device usually manages a plurality of disks (for example, 8 disks or 12 disks), and in order to ensure the performance of disk management, cache software running on the cache device discards the layer of the file system, manages the plurality of disks by adopting a bare disk management mode, and directly performs add-delete read-write on data in the disks.

In the related technology, a thread can be set in cache software to manage all disks, and the cache data adding and deleting operations of all disks are completed by one thread; the mode can ensure strict disk reading sequence; however, when the number of disks and the number of resource fragments are large, the method may cause performance bottleneck of cache software, resulting in low management efficiency of the disks. In order to avoid the above problem, all the fragments of the same resource may be stored on the same disk, but the disk may be biased, for example, when the reading frequency of a certain resource is high (the resource is referred to as a hot spot resource), the reading frequency of the disk where the resource is located is high, and other disks are rarely read, which is not favorable for the management of the disk.

In the related art, the cache software can also set a management thread for each disk. The multiple fragments of a resource are usually stored on different disks, and if the resource needs to be read, the management threads of the multiple disks that need to store the resource fragment all execute the task of reading the fragment. Because the resource version is not considered in the process of reading each resource fragment, if the version of the resource is updated in the reading process, the management threads of the multiple disks can easily read the resource fragments of different versions, so that the resource fragments of different versions are spliced together and then returned to the client, and the versions among the resource fragments are inconsistent, so that the resource reading error of the client is easily caused.

Based on this, the embodiment of the invention provides a resource acquisition method, a resource acquisition device and a server, which can be applied to resource reading processes of various distributed systems.

First, referring to a resource obtaining method shown in fig. 2, the method is applied to a node device of a CDN network; the method comprises the following steps:

step S200, according to a resource access request from a client, determining address information of a resource to be accessed and a fragment index number of first fragment data in the resource to be accessed; the resource access request carries address information of the resource to be accessed, and the resource to be accessed requested by the resource access request is allowed to be stored in a node device of the CDN in a segmented mode.

The Resource access request usually carries address information of the Resource to be accessed, such as a URL (Uniform Resource Locator) of a website storing the Resource to be accessed, e.g., www.xyz.com; the address information of the resource to be accessed can be obtained by analyzing from the resource access request data. The segment index number may indicate a position sequence of the segment data of the resource to be accessed in the segment data stored in the current device node, and may be obtained according to the related information of the resource to be accessed included in the resource access request, such as a data size of the resource to be accessed.

Step S202, according to the address information and the fragment index number of the first fragment data, the first fragment data in the resource to be accessed is obtained.

When the resources are stored in a fragmentation mode, a plurality of fragmentation data of one resource can be stored on a plurality of disks; when a storage disk is selected for each piece of fragmented data, a set rule is followed; when reading each piece of data of the resource, the corresponding storage disk of each piece of data can be found by using the set rule.

In practical application, a storage disk of the fragmented data can be selected according to a consistent hash (hash) principle; firstly, calculating the hash value of each disk according to the information such as the storage address, the serial number and the like of each disk, and mapping each disk on a hash ring; and calculating the hash value of each piece of sliced data according to the setting information of each piece of sliced data, mapping each piece of sliced data on the hash ring where the disk is located, and storing the piece of sliced data in the corresponding disk by adopting a proximity principle. The setting information of the fragment data may include address information (e.g., URL), fragment index number, and version information of the resource to which the fragment data belongs. The distribution of the fragmented data in the disk storage is performed through the consistent hash algorithm, so that the load balance of the disk can be realized, and the disk has uniqueness.

When the first fragment data is determined, the version information of the resource to be accessed in the current node equipment cannot be known, and default version information can be set for the first fragment data when the first fragment data is stored, so that a disk where the first fragment data is located can be quickly found according to the set rule according to the address information of the resource to be accessed, the fragment index number of the first fragment data and the default version information; and if the hash value is generated through the address information of the resource to be accessed, the fragment index number of the first fragment data and the default version information, determining the disk where the first fragment data is located.

After the disk where the first fragmented data is located is determined, the first fragmented data may be read from the disk. If the first fragment data is stored in the disk, acquiring the fragment data; if the first fragment data is not stored in the disk, the current node device may acquire a resource to be accessed from the node device on the previous layer; if the current node device is an edge node, the edge node may obtain the resource to be accessed from the middle node.

And step S204, determining the version information of the resource to be accessed through the first fragment data.

The version information is related to the number of times that the current node device performs source pulling on the resource to be accessed, and an initial version information is usually set for the resource to be accessed when the current node device stores the resource to be accessed for the first time; when the resource to be accessed is obtained by pulling back the source to the upper node equipment each time, the version information of the resource is changed, and if the source is pulled back once, the version information is added by one; the updated version information is stored in the first fragment data of the resource to be accessed, for example, the current version information of the resource to be accessed can be stored in the response header of the first fragment data; the version information of the resource to be accessed can be obtained by reading the version information data in the response header of the received fragment data.

Step S206, according to the address information, the fragment index number of the fragment data except the first fragment data in the resource to be accessed and the version information of the resource to be accessed, obtaining the fragment data except the first fragment data in the resource to be accessed, wherein the resource version indicated by the version information of the fragment data except the first fragment data in the resource to be accessed is the same as the resource version indicated by the version information of the first fragment data.

The above-mentioned segment index number of the segment data other than the first segment data may be determined according to the resource access request. If the resource to be accessed corresponding to the resource access request is a short video, the resource range of the short video can be acquired in the resource access request. When the fragmented data is stored in the disk, the respective fragmented data may be set to have the same size, for example, the data size of each fragmented data is set to be 1M. When the resource range of the short video is 1M to 5M, it can be determined that the respective segment index numbers are 1,2,3,4,5 (if the classification index numbers are set in a manner of sequentially increasing the arabic numbers).

The disk where the fragment data except the first fragment data is located is determined according to the resource to be accessed, the fragment index number and the version information of the resource to be accessed, so that the resource version indicated by the version information of the fragment data except the first fragment data stored in the determined disk is the same as the resource version indicated by the version information of the first fragment data; after determining the disk where each piece of fragmented data is located, the corresponding disk can be accessed to read the corresponding piece of fragmented data. When the disk does not have the fragment data stored therein, the current node device may acquire the resource to be accessed from the node device on the previous layer.

The current fragment data may be obtained by performing source back pulling on the current node device; and the resource to be accessed that is pulled back from the source may have changed. The data volume information of the resource to be accessed can be stored in each piece of fragmented data of the resource to be accessed, whether the data volume information of the resource to be accessed in the current piece of fragmented data and the current first piece of fragmented data is the same or not is compared, if the data volume information of the resource to be accessed in the current piece of fragmented data and the data volume information of the resource to be accessed in the current first piece of fragmented data are different, the resource to be accessed is changed, the acquisition process of the current piece of fragmented data is stopped, and all piece of fragmented data of the resource are deleted; and performing source returning pull on the whole resource to be accessed again, and responding to the resource access request again.

Step S208, the first fragment data and the fragment data other than the first fragment data are determined as the resource to be accessed corresponding to the acquired resource access request.

The resource to be accessed is stored in a disk of the current node device in a fragmentation storage mode, and when the fragment data of which the resource version is the same as the resource version indicated by the version information of the first fragment data of the resource to be accessed is obtained, the resource to be accessed corresponding to the resource access request is determined to be obtained; further, the resource to be accessed can also be returned to the client sending the resource access request.

The resource acquisition method is applied to node equipment of the CDN network; firstly, according to a resource access request from a client, determining address information of a resource to be accessed and a fragment index number of first fragment data in the resource to be accessed; after first fragment data in the resource to be accessed is obtained according to the address information and the fragment index number of the first fragment data, determining version information of the resource to be accessed through the first fragment data; and then, according to the address information, the fragment index number of the fragment data except the first fragment data in the resource to be accessed and the version information of the resource to be accessed, obtaining the fragment data except the first fragment data in the resource to be accessed. According to the method, the version information of the resource is determined through the first fragment data, and the fragment data except the first fragment data is obtained based on the version information, so that the consistency of the versions of the fragment data of the resource to be accessed is ensured, the accuracy and the stability of resource obtaining are improved, and the user experience is improved.

The embodiment of the invention also provides another resource acquisition method, which is realized on the basis of the method of the embodiment; the method mainly describes a specific implementation process (implemented by steps S300 and S302 below) for determining address information of a resource to be accessed and a segment index number of first segment data in the resource to be accessed according to a resource access request from a client, a specific implementation process (implemented by steps S304 to S308 below) for acquiring the first segment data in the resource to be accessed according to the address information and the segment index number of the first segment data, and a specific implementation process (implemented by steps S312 to S320 below) for acquiring segment data in the resource to be accessed except the first segment data; the method is applied to node equipment of the CDN network, wherein the node equipment comprises a plurality of disks; the disk is used for storing the fragment data; as shown in fig. 3, the method comprises the steps of:

step S300, extracting address information of the resource to be accessed and a resource range of the resource to be accessed indicated by the resource access request from the resource access request of the client; in practical implementation, the related information such as the data size of the resource to be accessed is stored in the resource access request, and a resource range of the resource to be accessed may be extracted from the resource access request, for example, the resource range of the resource to be accessed is the 1M th to the 5M th resource.

Step S302, acquiring the fragment index number of the first fragment data from the resource range; the fragment index number is related to the resource range and the data size of unit fragment data in the disk, and if the resource range is 1M to 5M and the unit data size stored in each fragment data is 1M, the fragment index number of the first fragment data is 1.

Step S304, calculating to obtain the keyword of the first fragment data according to the address information, the fragment index number of the first fragment data in the resource to be accessed and the preset default version information.

Since the first fragment data does not relate to the version problem of the resource to be accessed, default version information may be set for the first fragment data, for example, the default version information is set to 0. The keyword can be data information uniquely determined based on a set algorithm in combination with address information, a fragment index of first fragment data and default version information; after any one of the address information, the fragment index and the default version information is changed, the keyword is changed; and different address information, fragment indexes and default version information are combined based on a set algorithm, so that the same keywords cannot be obtained. The set algorithm may be a hash algorithm, and the obtained keyword is a hash value of the first piece of fragmented data.

And step S306, calculating to obtain a first target disk where the first fragment data is located by adopting a consistent hash algorithm according to the keyword of the first fragment data.

In the application process of the disk fragmentation storage, the consistent hash algorithm firstly calculates the hash value of each disk according to the information such as the storage address, the serial number and the like of each disk, and maps each disk on a hash ring. And when the keyword is the hash value of the first fragmented data, mapping the first fragmented data on the hash ring where the disk is located according to the hash value, and obtaining a first target disk where the first fragmented data is located by adopting a proximity principle. The method is realized on the premise that the disk is selected by the consistent hash algorithm in the storage process of the fragmented data of the resource to be accessed.

Step S308, the first fragment data in the resource to be accessed is obtained through the first target disk.

After determining the disk (i.e. the first target disk) where the first fragmented data is located, reading the first fragmented data from the disk; in the actual implementation process, each disk is provided with a thread, and the first fragment data can be read through the thread. In some cases, the fragment data may not be stored in the disk, and at this time, the current node device may obtain the resource to be accessed from the node device at the upper layer.

Step S310, determining the version information of the resource to be accessed through the first fragment data.

Step S312, extracting the resource range of the resource to be accessed from the resource access request.

Step S314, according to the data amount of the preset unit fragment data, obtaining the fragment index number of the fragment data except the first fragment data from the resource range.

The data volume of the unit fragment data is the maximum data volume which can be stored by each fragment data; if the data volume of one resource is 5.5M and the data volume of unit fragment data is 1M, 6 fragment data are needed for storing the resource, and the data volume of the last fragment data is 0.5M; if the resource range of one resource is from 1M to 5M, the segment index numbers of the segment data are 1,2,3,4,5 in sequence.

Step S316, aiming at each fragment data except the first fragment data in the resource to be accessed, calculating to obtain a keyword of the current fragment data according to the address information, the fragment index number of the current fragment data and the version information of the resource to be accessed; the process of calculating the keyword of the current fragment data is similar to the process of calculating the keyword of the first fragment data; in the actual implementation process, the calculation of the keywords of each piece of sliced data may be performed by parallel threads.

Step S318, calculating by using a consistent Hash algorithm according to the keywords of the current fragmented data to obtain a second target disk where the current fragmented data is located; this process is similar to the process of determining the target disk for the first fragmented data.

Step S320, obtaining the current fragment data through the second target disk.

In an actual implementation process, the current fragment data may be read by using a thread corresponding to the disk. In some cases, the fragment data may not be stored in the disk, and at this time, the current node device may obtain the resource to be accessed from the node device at the upper layer.

Step S322, determining the first fragment data and the fragment data other than the first fragment data as the resource to be accessed corresponding to the acquired resource access request.

The resource obtaining method comprises the steps of firstly obtaining keywords of first fragment data according to address information of resources to be accessed, fragment index numbers of the first fragment data and preset default version information, and determining a disk where the first fragment data is located by adopting a consistent hash algorithm according to the keywords; further acquiring first fragment data through the disk; after the version information of the resource to be accessed is obtained through the first fragment data, the keywords of each fragment data are calculated according to the address information, the fragment index numbers and the version information of each fragment data except the first fragment data, the disk where each fragment data is located is determined based on a consistent Hash algorithm, and then each fragment data is read. In the method, the disk where each piece of fragment data except the first piece of fragment data is located is determined based on the consistent hash algorithm and in combination with the version information of the resource to be accessed, so that the corresponding piece of fragment data is obtained, the consistency of each piece of fragment data of the resource to be accessed is ensured, and the accuracy of resource obtaining is improved.

The embodiment of the invention also provides another resource acquisition method, which is realized on the basis of the method of the embodiment; the method mainly describes a specific implementation process for determining the version information of the resource to be accessed through the first fragment data (which is implemented through the following steps S406-S414); as shown in fig. 4, the method includes the steps of:

step S400, according to the resource access request from the client, determining the address information of the resource to be accessed and the fragment index number of the first fragment data in the resource to be accessed.

Step S402, calculating to obtain the keyword of the first fragment data according to the address information, the fragment index number of the first fragment data in the resource to be accessed and the preset default version information.

And step S404, calculating by adopting a consistent hash algorithm according to the keyword of the first fragmented data to obtain a first target disk where the first fragmented data is located.

Step S406, inquiring whether the first fragment data in the resource to be accessed exists in the first target disk through the thread corresponding to the first target disk; if so, go to step S408; if not, go to step S412; in a specific implementation process, each disk corresponds to one thread, and the thread is used for managing fragment data in the disk corresponding to the thread; whether the first fragment data exists in the disk can be inquired by adopting a thread corresponding to the first target disk, and in the inquiry process, the corresponding data can be searched in the disk through the address information of the resource to be accessed, the classification index number of the first fragment data and other information.

Step S408, reading the first fragment data from the first target disk.

Step S410, reading the version information of the resource to be accessed from the response header of the first fragmented data, and executing step S416.

Step S412, acquiring the first fragment data from the upper device of the node device.

If the first fragment data in the resource to be accessed is not inquired in the first target disk, it is indicated that the resource to be accessed is not stored in the current edge node in advance; at this time, a request may be sent to a superior device of the current node device to obtain the first fragment data; the request usually includes address information of the resource to be accessed, a resource range, a fragment index number of the first fragment data, and other related information. If the node device is a device of an edge node, the upper level device of the node device may be a device of a middle level node, a device of an upper level node, or a device of a source station. If the node device is a device of a middle node, the upper level device of the node device may be a device of an upper node or a device of a source station. If the node device is a device of an upper node, the upper node device of the node device may be a device of a source station.

Step S414, setting preset initial version information for the resource to be accessed corresponding to the first fragmented data, taking the initial version information as the version information of the resource to be accessed, and adding the version information to the response header of the first fragmented data.

After the superior device sends the first fragment data to the current node device, the current node device sets preset initial version information for a resource to be accessed corresponding to the first fragment data; the initial version information may be a preset value, for example, the initial version information may be set to 0; the initial version information can be used as the version information of the resource to be accessed, and the version information is added to the response head of the first fragment data in a preset shelving manner; finally, the current node device may store the fragmented data in an internal disk according to a consistent hash algorithm.

Step S416, a resource range of the resource to be accessed is extracted from the resource access request.

Step S418, according to the data amount of the preset unit fragment data, obtain the fragment index number of the fragment data except the first fragment data from the resource range.

Step S420, for each piece of fragment data in the resource to be accessed except the first piece of fragment data, calculating to obtain a keyword of the current piece of fragment data according to the address information, the piece index number of the current piece of fragment data, and the read version information of the resource to be accessed.

Step S422, according to the keywords of the current fragmented data, a second target disk where the current fragmented data is located is obtained through calculation by adopting a consistent hash algorithm.

Step S424, querying whether the second target disk has current fragment data through a thread corresponding to the second target disk; if so, go to step S426; if not, go to step S428; the query process is similar to a process of querying whether the first fragment data in the resource to be accessed exists in the first target disk through a thread corresponding to the first target disk.

Step S426, the current fragment data is read from the second target disk, and step S430 is executed.

In step S428, the current fragment data is acquired from the upper device of the node device.

Step S430, determining the first fragment data and the fragment data except the first fragment data as the resource to be accessed corresponding to the acquired resource access request.

Specifically, if the current fragment data is obtained from the upper level device, the version of the resource to be accessed corresponding to the fragment data stored in the upper level device may be different from the version of the resource to be accessed stored in the current node device. For example, when the current node device is an upper node, the upper node device is a source station for storing resources to be accessed; the user uploading the resource to be accessed can modify the resource to be accessed stored in the source station, for example, 200M video resource is cut and edited, and 150M updated video resource is stored in the source station; at this time, the content of each piece of sliced data is changed; at this time, if the fragment data stored in the fragment data source acquired from the upper device and the other current node device is sent to the user as the resource to be accessed, a response error to the resource access request may be caused, and therefore, it is necessary to compare the version information of the current fragment data with the version information of the currently acquired first fragment data, as shown in fig. 5, the method specifically includes the following steps:

step S500, if the current fragment data is obtained from the superior device of the node device, extracting a first data size of the resource to be accessed from the current fragment data.

The first data size may be a data size of a resource to be accessed to which the current fragmented data belongs. The basic information of the resource to which the current fragment data belongs may be stored in the current fragment data, such as address information, data size, and the like of the resource, and the basic information may be used as a tag of the fragment data, so as to facilitate processing such as querying the fragment data.

Step S502, comparing whether the first data volume is the same as a second data volume of the resource to be accessed carried in the first fragment data of the resource to be accessed; if the two are the same, executing step S504; if not, go to step S510.

The second data size may be a data size of a resource to be accessed to which the first fragmented data belongs; when the first fragment data and the current fragment data belong to a resource to be accessed with the same version information, the first data volume and the second data volume are the same; if the two are different, it may be that the resource to be accessed of the source station is changed in the acquisition interval of the two fragmented data, that is, the first fragmented data and the current fragmented data, for example, the user who uploads the resource modifies the resource to be accessed.

Step S504, determining that the version of the current fragment data is the same as the version of the first fragment data of the resource to be accessed, and calculating to obtain the keyword of the current fragment data according to the address information, the fragment index number of the current fragment data and the version information of the resource to be accessed.

Step S506, according to the keywords of the current fragmented data, a disk for storing the current fragmented data is obtained by adopting a consistent Hash algorithm.

Step S508, the current fragment data is saved to the calculated disk. And (6) ending.

Step S510, determining that the current fragment data is different from the first fragment data of the resource to be accessed in version, and deleting the first fragment data of the resource to be accessed stored in the node device.

If the data volume of the resource to be accessed carried by the current fragment data and the first fragment data is different, it is indicated that the fragment data of the resource to be accessed stored in the current node device does not belong to the resource to be accessed of the same version, and if each fragment data is directly read, a response error to a resource access request is caused, so that the first fragment data of the resource to be accessed stored in the node device is deleted. After deleting the first fragment data of the resource to be accessed of the current node equipment, other data fragments which belong to the same resource version as the deleted first data fragment can not be accessed again; these no longer used data fragments are deleted during subsequent disk management.

Step S512, the first piece data of the resource to be accessed is obtained from the upper device of the node device again, the updated version information of the resource to be accessed is added to the response header of the first piece data, and the first piece data is stored.

After the resource access request is completed, the version information may be updated according to the number of times of pulling back the resource to be accessed by the current node device, the updated version information of the resource to be accessed is added to the response head of the first fragmented data, and each fragmented data is stored in the disk of the current node device according to the consistent hash algorithm.

Step S514, obtaining and storing the fragment data of the resource to be accessed, except the first fragment data, from the upper level device of the node device.

For the fragment data of the resource to be accessed, except for the first fragment data, after the fragment data is obtained from the upper-level device of the node device, the fragment data may be used as the current fragment data, and step S500 may be continuously performed until each fragment data of the resource to be accessed is stored to the disk.

In the resource obtaining method, in the process of obtaining the fragmented data of the resource to be accessed except the first fragmented data, whether the current fragmented data and the first fragmented data belong to the resource to be accessed in the same version is judged by comparing whether the data volume of the resource to be accessed indicated in the current fragmented data is the same as the data volume of the resource to be accessed indicated in the first fragmented data; if the current fragment data is the same as the current fragment data, reading the current fragment data; and if not, deleting the fragment data of the resource to be accessed of the current equipment node, and acquiring the fragment data of the resource to be accessed from the superior equipment of the node equipment to complete the resource access request of the resource to be accessed. In the method, in the acquisition process of the fragment data, whether each fragment data except the first fragment data and the first fragment data belong to the resource to be accessed of the same version or not is compared, so that the consistency of each fragment data of the resource to be accessed is ensured, and the accuracy of resource acquisition is improved.

The embodiment of the invention also provides another resource acquisition method, which is realized on the basis of the method of the embodiment; the method is applied to node equipment of the CDN; as shown in fig. 6, the node device of the CDN includes a Linux Virtual Server (LVS) and a plurality of cache devices (in fig. 5, two cache devices are included as an example); each cache device includes a kind of high-performance HTTP and reverse proxy web server and a cache disk (cache). The Linux virtual server can receive a resource access request sent by a user; and finally outputting the resource to be accessed corresponding to the resource access request from the cache disk through the function of the Nginx.

In the related art, a plurality of storage disks (simply referred to as disks) are provided in one cache device; each disk is managed by one thread; each resource is stored in a cache device in a fragmentation mode (also called block storage), and a disk is selected according to the principle of a consistent hash algorithm, namely a plurality of fragments of one resource can be stored on a plurality of disks. When fragment data of a resource to be accessed in a disk is read, a storage position of the fragment data is determined by only calculating a keyword (also called key) of the fragment data through address information (also called request URL) and a fragment index number in a resource access request, however, the consistency among a plurality of fragments in a multi-thread scene cannot be ensured through the method.

In the method, a disk is managed by adopting cache software, and as shown in FIG. 7, an architecture diagram of the cache software is shown; cache software comprises 3 threads which are a worker thread, a master thread and an io thread respectively; the master thread is responsible for configuration analysis of the worker thread and the io thread and can start the worker thread and the io thread; the worker thread is responsible for monitoring the port, acquiring the resource access request and processing service logic, for example, dividing one resource access request into a plurality of tasks (if n fragmented data are acquired, n tasks are divided), and each task is executed by one worker thread; the io threads are responsible for disk space management and provide functions of increasing, deleting, reading and writing and the like, and each io thread corresponds to one disk; the number of worker threads and the number of io threads are generally multiple, and the number of worker threads and the number of io threads are many-to-many access.

The method comprises the following steps:

(1) According to the (request URL + fragment index number + 0), calculating the key word of the first fragment data (also called the first fragment) of the resource to be accessed corresponding to the resource access request; specifically, because the response header and some key information are stored with the first fragmented data, the first fragmented data must be read first; since the first slice does not have a consistency problem, the version number of the first slice may be uniformly set to 0.

(2) And determining the disk to which the fragment belongs by adopting a consistent hash algorithm according to the keyword of the first fragment data.

(3) And inquiring whether the fragment data exists or not by an io thread to which the corresponding disk belongs.

(4) If the fragment data exists, a http (Hyper Text Transfer Protocol) response header of the fragment data is read from the disk, and version information (also called version number) of the resource is determined.

(5) If the fragment data does not exist, performing source return pulling on the upper layer node equipment; and after receiving the fragment data of the resource, generating version information of the resource, and storing the version information into an http response header of the first fragment data.

(6) And determining the fragmentation range of the fragmentation data of the required resource to be accessed according to the resource of the resource access request, thereby determining the fragmentation index number.

(7) Calculating keywords of each fragment data except the first fragment data according to the (request URL + fragment index number + version information); and determining the disk to which the fragmented data belongs by adopting a consistent hash algorithm according to the keywords of each fragmented data.

(8) And (5) repeating the steps (3) to (5) until all the fragment data of the resource to be accessed have responded to the client sending the resource access request.

If the related art is adopted to read the fragment data only by calculating the keywords of the fragment data through the address information (also called request URL) and the fragment index number in the resource access request, the following scenarios may occur:

(a) The worker thread 1 has successfully completed the task of creating the 1 st fragment data of the resource to be accessed, and the creating process is as described in the above step (1-5), and prepares to create the 2 nd fragment data of the resource to be accessed.

(b) The worker thread 2 does not find the 3 rd fragment data in the disk of the current node device, and requests the 3 rd fragment data from the source station; however, when the 3 rd fragment data is acquired, if it is found that the resource to be accessed in the source station has changed, all fragment data of the resource to be accessed are deleted in the current node device.

(c) And the worker thread 3 creates the 1 st fragment data of the resource to be accessed again, and prepares to create the 2 nd fragment data of the resource to be accessed after the creation.

(d) The worker thread 1 successfully creates the 2 nd fragment data of the resource to be accessed.

(e) The worker thread 3 creates the 2 nd fragment data of the resource to be accessed, and finds that the fragment data already exists and the creation fails. Fragment data created for the worker thread 1 is stored and used in the node device, but the 2 nd piece of data created by the worker thread 1 and the 1 st piece of data created by the worker thread 3 belong to resources to be accessed of different versions.

In the above scenario, the calculation that the version information is included in the keyword is not considered, so that when a resource access request is responded, fragments of different versions of the same resource are spliced together, and a response error occurs, such as a result that a video cannot be played.

However, according to the method provided by the embodiment of the present invention, version information is considered when calculating the cache keyword, and then the worker thread 3 in step (e) creates the 2 nd fragment data of the resource 1 successfully without being affected by the fragment data created by the worker thread 1. And the 2 nd fragment of the resource 1 created by the worker thread 1 cannot access the fragment data again in subsequent requests because the first fragment is deleted, and only the 2 nd fragment data created by the worker thread 3 is accessed, so that the final consistency among different fragments of the same resource is realized.

Although the method cannot guarantee the sequence of certain fragment operation of the resource to be accessed when responding to the resource access request, the final consistency of the resource can be guaranteed, namely, the fragments finally stored on different disks are the same version, and the contents of the versions cannot be sent to the client after being spliced.

Corresponding to the embodiment of the resource obtaining method, the embodiment of the invention also provides a resource obtaining device which is arranged at the node equipment of the CDN network; as shown in fig. 8, the apparatus includes:

the fragment index number determining module 700 is configured to determine, according to a resource access request from a client, address information of a resource to be accessed and a fragment index number of first fragment data in the resource to be accessed, where the resource access request carries the address information of the resource to be accessed, and the resource to be accessed requested by the resource access request is allowed to be fragmented and stored in node devices of the CDN network;

a first fragment data obtaining module 702, configured to obtain a first fragment data in a resource to be accessed according to the address information and a fragment index number of the first fragment data;

a version information determining module 704, configured to determine, through the first fragment data, version information of a resource to be accessed;

a second fragment data obtaining module 706, configured to obtain the fragment data of the resource to be accessed except the first fragment data according to the address information, the fragment index number of the fragment data of the resource to be accessed except the first fragment data, and the version information of the resource to be accessed, where a resource version indicated by the version information of the fragment data of the resource to be accessed except the first fragment data is the same as a resource version indicated by the version information of the first fragment data;

the to-be-accessed resource determining module 708 is configured to determine the first fragment data and the fragment data other than the first fragment data as the to-be-accessed resource corresponding to the acquired resource access request.

The resource acquisition device is applied to node equipment of a CDN network; firstly, according to a resource access request from a client, determining address information of a resource to be accessed and a fragment index number of first fragment data in the resource to be accessed; after first fragment data in the resource to be accessed is obtained according to the address information and the fragment index number of the first fragment data, determining version information of the resource to be accessed through the first fragment data; and then obtaining the fragment data except the first fragment data in the resource to be accessed according to the address information, the fragment index number of the fragment data except the first fragment data in the resource to be accessed and the version information of the resource to be accessed. According to the method, the version information of the resource is determined through the first fragment data, and the fragment data except the first fragment data is obtained based on the version information, so that the consistency of the versions of the fragment data of the resource to be accessed is ensured, the accuracy and the stability of resource obtaining are improved, and the user experience is improved.

Specifically, the segment index number determining module is further configured to: extracting address information of the resource to be accessed and a resource range of the resource to be accessed indicated by the resource access request from the client; and acquiring the fragment index number of the first fragment data from the resource range.

In general, the node device includes a plurality of disks; the disk is used for storing the fragment data; the first sliced data obtaining module is further configured to: calculating to obtain a keyword of the first fragment data according to the address information, the fragment index number of the first fragment data in the resource to be accessed and preset default version information; calculating by adopting a consistent hash algorithm according to the keyword of the first fragmented data to obtain a first target disk where the first fragmented data is located; and acquiring the first fragment data in the resource to be accessed through the first target disk.

In actual implementation, each disk corresponds to one thread, and the thread is used for managing fragment data in the disk corresponding to the thread; the first sliced data obtaining module is further configured to: inquiring whether the first fragment data in the resource to be accessed exists in the first target disk through the thread corresponding to the first target disk; under the condition that first fragment data in resources to be accessed exist in a first target disk, reading the first fragment data from the first target disk; and under the condition that the first fragment data in the resource to be accessed does not exist in the first target disk, acquiring the first fragment data from the superior equipment of the node equipment.

Specifically, the version information determining module is further configured to: if the first fragment data is acquired from the node equipment, reading the version information of the resource to be accessed from the response head of the first fragment data; if the first fragment data is acquired from the superior device of the node device, preset initial version information is set for the resource to be accessed corresponding to the first fragment data, and the initial version information is used as the version information of the resource to be accessed and is added to a response header of the first fragment data.

Specifically, the second sliced data obtaining module is further configured to: extracting a resource range of a resource to be accessed from the resource access request; acquiring the fragment index numbers of the fragment data except the first fragment data from the resource range according to the data volume of the preset unit fragment data; aiming at each fragment data except the first fragment data in the resource to be accessed, calculating to obtain a keyword of the current fragment data according to the address information, the fragment index number of the current fragment data and the version information of the resource to be accessed; calculating by adopting a consistent hash algorithm according to the keywords of the current fragmented data to obtain a second target disk where the current fragmented data is located; and acquiring the current fragment data through the second target disk.

Further, the second sliced data obtaining module is further configured to: the step of obtaining the current fragment data through the second target disk includes: inquiring whether the current fragment data exists in the second target disk through a thread corresponding to the second target disk; if yes, reading the current fragment data from the second target disk; and if the current fragment data does not exist, acquiring the current fragment data from the superior device of the node device.

Further, the above apparatus further comprises: the second sliced data acquisition module is further configured to: the first data volume extraction module is used for extracting a first data volume of the resource to be accessed from the current fragment data; the comparison module is used for comparing whether the first data volume is the same as the second data volume of the resource to be accessed carried in the first fragment data of the resource to be accessed; the keyword calculation module is used for determining that the version of the current fragment data is the same as that of the first fragment data of the resource to be accessed if the version of the current fragment data is the same as that of the first fragment data of the resource to be accessed, and calculating to obtain a keyword of the current fragment data according to the address information, the fragment index number of the current fragment data and the version information of the resource to be accessed; the disk determining module is used for calculating to obtain a disk for storing the current fragmented data by adopting a consistent Hash algorithm according to the keywords of the current fragmented data; and the fragment data storage module is used for storing the current fragment data to the calculated disk.

Further, the above apparatus further comprises: the fragment data deleting module is used for determining that the current fragment data is different from the first fragment data of the resource to be accessed in version if the first data volume is different from the second data volume, and deleting the first fragment data of the resource to be accessed stored in the node equipment; the fragment data reacquiring module is used for reacquiring the first fragment data of the resource to be accessed from the superior equipment of the node equipment, adding the updated version information of the resource to be accessed to the response header of the first fragment data, and storing the first fragment data; and acquiring and storing the fragment data of the resource to be accessed except the first fragment data from the superior equipment of the node equipment.

The resource obtaining apparatus provided in the embodiment of the present invention has the same implementation principle and technical effect as those of the foregoing resource obtaining method embodiment, and for brief description, reference may be made to corresponding contents in the foregoing resource obtaining method embodiment for a part not mentioned in the embodiment of the resource obtaining apparatus.

An embodiment of the present invention further provides a server, as shown in fig. 9, where the server includes aprocessor 130 and amemory 131, thememory 131 stores machine executable instructions capable of being executed by theprocessor 130, and theprocessor 130 executes the machine executable instructions to implement the resource obtaining method.

Further, the server shown in fig. 9 further includes a bus 132 and acommunication interface 133, and theprocessor 130, thecommunication interface 133 and thememory 131 are connected through the bus 132.

TheMemory 131 may include a Random Access Memory (RAM) and a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 133 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used. The bus 132 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 9, but this does not indicate only one bus or one type of bus.

Theprocessor 130 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in theprocessor 130. TheProcessor 130 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in ram, flash, rom, prom, or eprom, registers, etc. as is well known in the art. The storage medium is located in thememory 131, and theprocessor 130 reads the information in thememory 131 and completes the steps of the method of the foregoing embodiment in combination with the hardware thereof.

The embodiment of the present invention further provides a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement the resource acquisition method.

The resource acquisition method and apparatus provided in the embodiments of the present invention, and the computer program product of the server include a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the method described in the foregoing method embodiments, and specific implementation may refer to the method embodiments, which are not described herein again.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (12)

1. A resource obtaining method is characterized in that the method is applied to node equipment of a CDN network, wherein the node equipment is used for managing a plurality of disks, and the disks are used for storing fragment data; the method comprises the following steps:

according to a resource access request from a client, determining address information of a resource to be accessed and a fragment index number of first fragment data in the resource to be accessed;

acquiring the first fragment data in the resource to be accessed according to the address information and the fragment index number of the first fragment data; when the fragment data is acquired, determining a storage disk corresponding to the acquired fragment data by using a rule followed when a storage disk is selected for each fragment data;

determining the version information of the resource to be accessed according to the first fragment data; the version information is determined based on a response header field in the first fragmented data;

acquiring the fragmented data of the resource to be accessed except the first fragmented data according to the address information, the fragmented index number of the fragmented data of the resource to be accessed except the first fragmented data and the version information of the resource to be accessed, wherein the resource version indicated by the version information of the fragmented data of the resource to be accessed except the first fragmented data is the same as the resource version indicated by the version information of the first fragmented data;

and determining the first fragment data and the fragment data except the first fragment data as the resource to be accessed corresponding to the acquired resource access request.

2. The method according to claim 1, wherein the step of determining address information of the resource to be accessed and a segment index number of a first segment data in the resource to be accessed according to a resource access request from a client comprises:

extracting address information of a resource to be accessed and a resource range of the resource to be accessed indicated by a resource access request from a client;

and acquiring the fragment index number of the first fragment data from the resource range.

3. The method of claim 1, wherein the node device includes a plurality of disks; the disk is used for storing the fragment data; the step of obtaining the first fragment data in the resource to be accessed according to the address information and the fragment index number of the first fragment data includes:

calculating to obtain a keyword of the first fragment data according to the address information, the fragment index number of the first fragment data in the resource to be accessed and preset default version information;

calculating by adopting a consistent hash algorithm according to the keyword of the first fragmented data to obtain a first target disk where the first fragmented data is located;

and acquiring the first fragment data in the resource to be accessed through the first target disk.

4. The method according to claim 3, wherein each disk corresponds to a thread, and the thread is used for managing the fragment data in the disk corresponding to the thread;

the step of obtaining the first fragment data in the resource to be accessed through the first target disk includes:

inquiring whether the first fragment data in the resource to be accessed exists in the first target disk or not through a thread corresponding to the first target disk;

under the condition that the first fragmented data in the resource to be accessed exists in the first target disk, reading the first fragmented data from the first target disk;

and acquiring the first fragment data from the superior device of the node device under the condition that the first fragment data in the resource to be accessed does not exist in the first target disk.

5. The method according to claim 1, wherein the step of determining the version information of the resource to be accessed through the first fragment data comprises:

under the condition that the first fragment data is acquired from the node equipment, reading the version information of the resource to be accessed from a response head of the first fragment data;

and under the condition that the first fragment data is acquired from the superior device of the node device, setting preset initial version information for the resource to be accessed corresponding to the first fragment data, taking the initial version information as the version information of the resource to be accessed, and adding the version information to a response header of the first fragment data.

6. The method according to claim 1, wherein the step of obtaining the fragment data of the resource to be accessed other than the first fragment data according to the address information, the fragment index number of the fragment data of the resource to be accessed other than the first fragment data, and the version information of the resource to be accessed comprises:

extracting the resource range of the resource to be accessed from the resource access request;

acquiring the fragment index numbers of the fragment data except the first fragment data from the resource range according to the data volume of the preset unit fragment data;

for each fragment data except the first fragment data in the resource to be accessed, calculating to obtain a keyword of the current fragment data according to the address information, the fragment index number of the current fragment data and the version information of the resource to be accessed;

calculating by adopting a consistent hash algorithm according to the keywords of the current fragmented data to obtain a second target disk where the current fragmented data is located;

and acquiring the current fragment data through the second target disk.

7. The method of claim 6, wherein the step of obtaining the current fragmented data via the second target disk comprises:

inquiring whether the current fragment data exists in the second target disk through a thread corresponding to the second target disk;

reading the current fragment data from the second target disk under the condition that the current fragment data exists in the second target disk;

and acquiring the current fragment data from a superior device of the node device under the condition that the current fragment data does not exist in the second target disk.

8. The method according to claim 7, wherein after the step of obtaining the current fragmented data from a superior device of the node device, the method further comprises:

extracting a first data size of the resource to be accessed from the current fragment data;

comparing whether the first data volume is the same as a second data volume of the resource to be accessed carried in the first fragment data of the resource to be accessed;

if the current fragment data is the same as the first fragment data of the resource to be accessed, determining that the version of the current fragment data is the same as the version of the first fragment data of the resource to be accessed, and calculating to obtain a keyword of the current fragment data according to the address information, the fragment index number of the current fragment data and the version information of the resource to be accessed;

according to the keywords of the current fragmented data, a disk for storing the current fragmented data is obtained by adopting a consistent hash algorithm;

and storing the current fragment data to the calculated disk.

9. The method of claim 8, further comprising:

if the first data volume is different from the second data volume, determining that the current fragment data is different from the first fragment data of the resource to be accessed in version, and deleting the first fragment data of the resource to be accessed stored in the node equipment;

acquiring the first fragment data of the resource to be accessed from the superior device of the node device again, adding the updated version information of the resource to be accessed to the response head of the first fragment data, and storing the first fragment data;

and acquiring and storing the fragment data of the resource to be accessed except the first fragment data from the superior device of the node device.

10. The resource acquisition device is characterized by being arranged on node equipment of a CDN (content delivery network), wherein the node equipment is used for managing a plurality of disks, and the disks are used for storing fragment data; the device comprises:

the system comprises a fragment index number determining module, a fragment index number determining module and a fragment index number determining module, wherein the fragment index number determining module is used for determining address information of a resource to be accessed and a fragment index number of first fragment data in the resource to be accessed according to a resource access request from a client;

a first fragment data obtaining module, configured to obtain the first fragment data in the resource to be accessed according to the address information and the fragment index number of the first fragment data; when the fragment data is acquired, determining a storage disk corresponding to the acquired fragment data by using a rule followed when a storage disk is selected for each fragment data;

the version information determining module is used for determining the version information of the resource to be accessed through the first fragment data; the version information is determined based on a response header field in the first fragmented data;

a second fragment data obtaining module, configured to obtain, according to the address information, a fragment index number of fragment data in the resource to be accessed except the first fragment data, and version information of the resource to be accessed, fragment data in the resource to be accessed except the first fragment data, where a resource version indicated by the version information of the fragment data in the resource to be accessed except the first fragment data is the same as a resource version indicated by the version information of the first fragment data;

and the to-be-accessed resource determining module is used for determining the first fragment data and the fragment data except the first fragment data as the to-be-accessed resource corresponding to the acquired resource access request.

11. A server comprising a processor and a memory, the memory storing machine executable instructions executable by the processor, the processor executing the machine executable instructions to implement the method of any one of claims 1 to 9.

12. A machine-readable storage medium having stored thereon machine-executable instructions which, when invoked and executed by a processor, cause the processor to perform the method of any of claims 1 to 9.

Images