CN114442959A - Data writing method, device and system for multi-region storage system - Google Patents
Data writing method, device and system for multi-region storage systemDownload PDFInfo
- Publication number
- CN114442959A CN114442959ACN202210110211.5ACN202210110211ACN114442959ACN 114442959 ACN114442959 ACN 114442959ACN 202210110211 ACN202210110211 ACN 202210110211ACN 114442959 ACN114442959 ACN 114442959A
- Authority
- CN
- China
- Prior art keywords
- data
- partition
- storage
- server
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Translated fromChinese技术领域technical field
本公开涉及人工智能技术中的大数据、信息流、以及云计算，尤其涉及一种多地域存储系统的数据写入方法、装置及系统。The present disclosure relates to big data, information flow, and cloud computing in artificial intelligence technology, and in particular, to a data writing method, device, and system for a multi-region storage system.
背景技术Background technique
多地域存储系统是指，基于部署于多个地域的服务器而形成的对数据进行存储和处理的系统，而如何将各地域的服务器中的数据进行同步成了亟待解决的问题。A multi-regional storage system refers to a system for storing and processing data based on servers deployed in multiple regions, and how to synchronize data in servers in various regions has become an urgent problem to be solved.
在相关技术中，可以采用“多地域多写”的方式实现，例如，若某地域的服务器接收到待写入数据，则将该待写入数据同步给其他地域的服务器，以使得各地域的服务器完成写入待写入数据的操作。In the related art, the method of "multiple regions and multiple writes" can be adopted. For example, if a server in a certain region receives data to be written, the to-be-written data is synchronized to servers in other regions, so that the The server completes the operation of writing the data to be written.
然而，采用上述方法，多地域存储系统的存储稳定性与地域的数量相关，尤其当地域的数量较多时，存在多地域存储系统的存储稳定性偏低的技术问题。However, with the above method, the storage stability of the multi-region storage system is related to the number of regions, especially when the number of regions is large, there is a technical problem that the storage stability of the multi-region storage system is low.
发明内容SUMMARY OF THE INVENTION
本公开提供了一种用于提高数据存储稳定性的多地域存储系统的数据写入方法、装置及系统。The present disclosure provides a data writing method, device and system for a multi-region storage system for improving data storage stability.
根据本公开的第一方面，提供了一种多地域存储系统的数据写入方法，包括：According to a first aspect of the present disclosure, a data writing method for a multi-region storage system is provided, including:
获取写入请求，其中，所述写入请求中携带待写入数据；obtaining a write request, wherein the write request carries data to be written;
对所述待写入数据进行分区处理，得到所述待写入数据的分区区域中的分区数据，并将所述待写入数据的分区区域中的分区数据存储至预设队列中；Partition processing is performed on the data to be written to obtain the partition data in the partition area of the data to be written, and the partition data in the partition area of the data to be written is stored in a preset queue;
控制多地域存储系统中各地域的服务器，从所述预设队列中读取并存储所述待写入数据的分区区域中的分区数据。The server in each region in the multi-region storage system is controlled, and the partition data in the partition region where the data to be written is read and stored from the preset queue.
根据本公开的第二方面，提供了一种一种多地域存储系统的数据写入装置，包括：According to a second aspect of the present disclosure, a data writing device of a multi-region storage system is provided, comprising:
获取单元，用于获取写入请求，其中，所述写入请求中携带待写入数据；an obtaining unit, configured to obtain a write request, wherein the write request carries data to be written;
分区单元，用于对所述待写入数据进行分区处理，得到所述待写入数据的分区区域中的分区数据；a partition unit, configured to perform partition processing on the data to be written to obtain partition data in the partition area of the data to be written;
存储单元，用于将所述待写入数据的分区区域中的分区数据存储至预设队列中；a storage unit for storing the partition data in the partition area of the data to be written into a preset queue;
控制单元，用于控制多地域存储系统中各地域的服务器，从所述预设队列中读取并存储所述待写入数据的分区区域中的分区数据。The control unit is configured to control the servers in each region in the multi-region storage system to read and store the partition data in the partition region of the data to be written from the preset queue.
根据本公开的第三方面，提供了一种电子设备，包括：According to a third aspect of the present disclosure, there is provided an electronic device, comprising:
至少一个处理器；以及at least one processor; and
与所述至少一个处理器通信连接的存储器；其中，a memory communicatively coupled to the at least one processor; wherein,
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行第一方面所述的方法。The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of the first aspect.
根据本公开的第四方面，提供了一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行根据第一方面所述的方法。According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method according to the first aspect.
根据本公开的第五方面，提供了一种计算机程序产品，所述计算机程序产品包括：计算机程序，所述计算机程序存储在可读存储介质中，电子设备的至少一个处理器可以从所述可读存储介质读取所述计算机程序，所述至少一个处理器执行所述计算机程序使得电子设备执行第一方面所述的方法。According to a fifth aspect of the present disclosure, there is provided a computer program product, the computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can Reading the storage medium reads the computer program, and executing the computer program by the at least one processor causes the electronic device to perform the method of the first aspect.
根据本公开的第六方面，提供了一种多地域存储系统，所述多地域存储系统通过如第一方面所述的方法存储待写入数据，所述多地域存储系统包括：各地域各自对应的服务器，每一地域的服务器包括存储矩阵，所述存储矩阵包括多个存储引擎。According to a sixth aspect of the present disclosure, a multi-regional storage system is provided. The multi-regional storage system stores data to be written by using the method according to the first aspect. The multi-regional storage system includes: each region corresponds to The server in each region includes a storage matrix, and the storage matrix includes a plurality of storage engines.
根据本公开的第七方面，提供了一种多地域存储系统的数据写入系统，包括：According to a seventh aspect of the present disclosure, a data writing system of a multi-region storage system is provided, including:
如第二方面所述的装置；The device of the second aspect;
如第六方面所述的多地域存储系统。The multi-region storage system according to the sixth aspect.
根据本公开的通过分区处理得到分区数据，并将分区数据存储至预设队列，以控制各地域的服务器从预设队列中获取并写入分区数据的技术特征，可以实现各地域的服务器的同步写入，避免了各地域的服务器所存储的数据不一致的弊端，提高了数据写入的可靠性和准确性，也提高了多地域存储系统的存储稳定性的技术效果。According to the technical features of the present disclosure, the partition data is obtained by partition processing, and the partition data is stored in a preset queue, so as to control the servers in each region to obtain and write the partition data from the preset queue, so that the synchronization of the servers in each region can be realized. Writing, avoids the disadvantage of inconsistent data stored by servers in different regions, improves the reliability and accuracy of data writing, and also improves the technical effect of storage stability of the multi-region storage system.
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。It should be understood that what is described in this section is not intended to identify key or critical features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.
附图说明Description of drawings
附图用于更好地理解本方案，不构成对本公开的限定。其中：The accompanying drawings are used for better understanding of the present solution, and do not constitute a limitation to the present disclosure. in:
图1是根据本公开第一实施例的示意图；1 is a schematic diagram according to a first embodiment of the present disclosure;
图2是根据本公开第二实施例的示意图；2 is a schematic diagram according to a second embodiment of the present disclosure;
图3是根据本公开实施例的地域分布的示意图；3 is a schematic diagram of geographic distribution according to an embodiment of the present disclosure;
图4是本公开实施例的原理示意图；4 is a schematic diagram of the principle of an embodiment of the present disclosure;
图5是根据本公开第三实施例的示意图；5 is a schematic diagram according to a third embodiment of the present disclosure;
图6是根据本公开第四实施例的示意图；6 is a schematic diagram according to a fourth embodiment of the present disclosure;
图7是根据本公开第五实施例的示意图；7 is a schematic diagram according to a fifth embodiment of the present disclosure;
图8是用来实现本公开实施例的多地域存储系统的数据写入方法的电子设备的框图。FIG. 8 is a block diagram of an electronic device used to implement the data writing method of the multi-region storage system according to the embodiment of the present disclosure.
具体实施方式Detailed ways
以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本公开的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.
数据是互联网企业的核心内容，存储系统是对数据进行处理的核心技术之一。随着互联网技术的发展以及信息的多元化，存储系统由单一地域的存储系统发展为多地域存储系统。Data is the core content of Internet enterprises, and storage system is one of the core technologies for data processing. With the development of Internet technology and the diversification of information, the storage system has developed from a single-regional storage system to a multi-regional storage system.
其中，多地域存储系统中包括多个地域的服务器，针对任一地域，该地域的服务器可以为具有数据存储和处理功能的服务器，数据处理功能包括数据访问、写入、读取、以及修改等功能，且该服务器可以为独立的服务器，也可以为由多个服务器组成的服务器集群，可以为虚拟服务器，也可以为物理服务器，也可以为云平台等，本实施例不做限定。Among them, the multi-regional storage system includes servers in multiple regions. For any region, the server in this region can be a server with data storage and processing functions. The data processing functions include data access, writing, reading, and modification, etc. function, and the server may be an independent server, a server cluster composed of multiple servers, a virtual server, a physical server, or a cloud platform, which is not limited in this embodiment.
为了实现多地域存储系统中各地域的服务器中的数据的一致性，需要将待写入数据同步写入至各地域的服务器中。In order to achieve the consistency of the data in the servers in each region in the multi-region storage system, the data to be written needs to be synchronously written to the servers in each region.
一个示例中，在接收到待写入数据时，可以将待写入数据传输给各地域的服务器，以使得各地域的服务器完成写入待写入数据的操作。In one example, when the data to be written is received, the data to be written may be transmitted to the servers in each region, so that the servers in each region complete the operation of writing the data to be written.
例如，可以将待写入数据传输给各地域的服务器各自配置的数据库存储引擎，以便由各地域的服务器各自配置的数据库存储引擎对待写入数据进行写入处理。For example, the data to be written can be transmitted to the database storage engines configured by the servers in each region, so that the data to be written can be written by the database storage engines configured by the servers in each region.
另一个示例中，在接收到待写入数据时，可以基于预先设置的地域写入需求，将待写入数据传输给与地域写入需求对应的地域的服务器，以完成待写入数据的写入。In another example, when the data to be written is received, the data to be written may be transmitted to the server in the region corresponding to the regional write demand based on the preset regional write demand, so as to complete the writing of the to-be-written data. enter.
然而，采用上述方法，多地域存储系统的存储稳定性是每个地域的存储稳定性的乘积，随着地域数量的增加，多地域存储系统的存储稳定性可能成指数下降，无法满足稳定性要求。However, with the above method, the storage stability of a multi-regional storage system is the product of the storage stability of each region. As the number of regions increases, the storage stability of a multi-regional storage system may decrease exponentially, failing to meet the stability requirements. .
再一个示例中，在多地域存储系统的各地域中，确定主地域和从地域，部署于主地域的服务器为主服务器，部署于从地域的服务器为从服务器，在接收到待写入数据时，先将待写入数据写入至主地域的主服务器，而后由主地域的主服务器将待写入数据同步至从地域的从服务器，以完成待写入数据在多地域存储系统中的写入。In another example, in each region of the multi-region storage system, the master region and the slave region are determined, the server deployed in the master region is the master server, and the server deployed in the slave region is the slave server. , first write the data to be written to the master server in the master region, and then the master server in the master region synchronizes the data to be written to the slave server in the slave region to complete the writing of the data to be written in the multi-region storage system enter.
然而，采用该方法，待写入数据被写入至多地域存储系统中的各地域的服务器的时序性无法保证，容易存在各地域的服务器中存储的待写入数据不一致的问题。However, with this method, the timing of writing data to be written to servers in different regions in the multi-region storage system cannot be guaranteed, and the problem of inconsistency of data to be written stored in servers in different regions is likely to occur.
为了避免上述技术问题中的至少一种，本公开发明人经过创造性的劳动，得到了本公开的发明构思：将待写入数据进行分区处理，得到多个分区区域，每个分区区域中有待写入数据的部分数据(即分区数据)，将各分区数据存储至预设队列，以使得多地域存储系统中各地域的服务器，从预设队列中读取并存储各分区数据，从而完成多地域存储系统对待写入数据的写入处理。In order to avoid at least one of the above technical problems, the inventor of the present disclosure has obtained the inventive concept of the present disclosure through creative work: partitioning the data to be written to obtain a plurality of partitioned areas, each of which is to be written in the partitioned area. Part of the data (that is, partition data) of the input data, and store the data of each partition in the preset queue, so that the servers in each region in the multi-region storage system can read and store the data of each partition from the preset queue, so as to complete the multi-regional storage system. The storage system writes the data to be written.
基于上述发明构思，本公开提供一种多地域存储系统的数据写入方法、装置及系统，应用于人工智能技术中的大数据、信息流、以及云计算，以达到提高多地域存储系统的存储稳定性。Based on the above inventive concept, the present disclosure provides a data writing method, device and system for a multi-regional storage system, which are applied to big data, information flow, and cloud computing in artificial intelligence technology, so as to improve the storage capacity of the multi-regional storage system. stability.
图1是根据本公开第一实施例的示意图，如图1所示，本公开实施例的多地域存储系统的数据写入方法，包括：FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure. As shown in FIG. 1 , a data writing method of a multi-region storage system according to an embodiment of the present disclosure includes:
S101：获取写入请求，其中，写入请求中携带待写入数据。S101: Obtain a write request, where the write request carries data to be written.
示例性的，本实施例的执行主体可以为多地域存储系统的数据写入装置(下文简称为写入装置)，写入装置可以为服务器(如本地服务器，或者，云端服务器，也可以为服务器集群等)，也可以为终端设备，也可以为处理器，也可以为芯片等，本实施例不做限定。Exemplarily, the execution body of this embodiment may be a data writing device (hereinafter referred to as a writing device) of a multi-region storage system, and the writing device may be a server (such as a local server, or a cloud server, or a server). Cluster, etc.), may also be a terminal device, may also be a processor, may also be a chip, etc., which is not limited in this embodiment.
待写入数据是指，需要被写入至多地域存储系统中的数据。待写入数据基于本公开实施例的多地域存储系统的数据写入方法的应用场景不同而不同。例如，若本公开实施例的多地域存储系统的数据写入方法应用于论坛场景，则待写入数据可以为论坛中的发帖内容、帖子的回复内容等。The data to be written refers to the data that needs to be written to the multi-region storage system. The data to be written varies based on the application scenarios of the data writing method of the multi-region storage system according to the embodiment of the present disclosure. For example, if the data writing method of the multi-region storage system according to the embodiment of the present disclosure is applied to a forum scenario, the data to be written may be the post content in the forum, the reply content of the post, and the like.
其中，至少可以采用下述方法获取写入请求：Among them, at least the following methods can be used to obtain the write request:
一个示例中，写入请求可以为用户通过用户设备(如移动终端等)向写入装置发起的，且写入请求可以为语音指令形式的请求，也可以为触控形式的请求，本实施例不做限定。In an example, the writing request may be initiated by the user to the writing device through the user equipment (such as a mobile terminal, etc.), and the writing request may be a request in the form of a voice command, or a request in the form of touch. This embodiment Not limited.
另一个示例中，写入装置可以对是否有写入请求进行监听，如可以采用实时监听的方式获取写入请求，也可以基于时间间隔监听的方式获取写入请求。In another example, the writing device may monitor whether there is a write request. For example, the write request may be obtained by monitoring in real time, or the write request may be obtained by monitoring at a time interval.
S102：对待写入数据进行分区(partition)处理，得到待写入数据的分区区域中的分区数据，并将待写入数据的分区区域中的分区数据存储至预设队列(queue)中。S102: Perform partition processing on the data to be written, obtain the partition data in the partition area of the data to be written, and store the partition data in the partition area of the data to be written in a preset queue (queue).
示例性的，将待写入数据划分为一个或多个(通常为多个)区域，将划分的区域称为分区区域，区分区域中至少包括待写入数据的至少部分数据(即分区数据)。Exemplarily, the data to be written is divided into one or more (usually multiple) areas, the divided areas are called partition areas, and the divided areas at least include at least part of the data to be written (ie, partition data) .
例如，若分区区域为一个，则该分区区域中的分区数据即为待写入数据；若分区区域为N(N为大于1的正整数)个，则每一分区区域中的分区数据为待写入数据中的部分数据，N个分区区域中的所有分区数据为待写入数据。For example, if there is one partition area, the partition data in the partition area is the data to be written; if the partition area is N (N is a positive integer greater than 1), the partition data in each partition area is the data to be written. Part of the data in the written data, all the partition data in the N partition areas are the data to be written.
本实施例对分区处理的方式不做限定，且对分区区域的数量不做限定。例如，可以由写入装置基于需求、历史记录、以及试验等方式确定分区区域的数量，也可以结合写入装置运行的进程的数量确定分区区域的数量；可以结合分区区域的数量确定分区处理的方法或参数。This embodiment does not limit the manner of partition processing, and does not limit the number of partition regions. For example, the number of partitioned areas can be determined by the writing device based on requirements, historical records, and experiments, or the number of partitioned areas can be determined in combination with the number of processes running by the writing device; the number of partitioned areas can be determined in combination with the number of partitioned areas method or parameter.
队列是一种特殊的线性表，特殊之处在于它允许在表的前端(front)进行删除操作，而在表的后端(rear)进行插入操作，A queue is a special kind of linear table. The special feature is that it allows delete operations to be performed at the front of the table and insert operations to be performed at the rear of the table.
本实施例中，对队列的类型不做限定。例如，队列可以为“先进先出”策略的队列，可以为消息队列利器(Mq-proxy)。In this embodiment, the type of the queue is not limited. For example, the queue can be a queue with a "first-in, first-out" policy, and can be a message queue tool (Mq-proxy).
S103：控制多地域存储系统中各地域的服务器，从预设队列中读取并存储待写入数据的分区区域中的分区数据。S103: Control the servers in each region in the multi-region storage system, and read and store the partition data in the partition region where the data to be written is read from the preset queue.
示例性的，若多地域存储系统中的多地域包括M(M为大于1的正整数)个地域，则M个地域中的每个地域的服务器，均从预设队列中读取并写入分区数据，以实现对待写入数据的写入。Exemplarily, if the multi-regions in the multi-region storage system include M (M is a positive integer greater than 1) regions, the servers in each region in the M regions will read and write from the preset queue. Partition the data to implement the writing of the data to be written.
在本实施例中，由于各地域的服务器均从预设队列中读取并写入待写入数据，可以避免相关技术中因主地域的服务器写入后，同步至从地域的服务器而造成的各地域的服务器的写入操作的时序性无法保证，造成各地域的服务器中的数据不一致的弊端，提高了各地域的服务器中的数据的一致性，从而实现在对被存储的数据的应用时，如浏览上述示例中的论坛内容时，具有较高的时限性和有效性的技术效果；且可以避免随着地域数量的增加，多地域存储系统的存储稳定性可能成指数下降的弊端，提高多地域存储的存储稳定性的技术效果。In this embodiment, since the servers in each region all read and write the data to be written from the preset queue, it is possible to avoid the problem caused by the synchronization of the server in the master region to the server in the slave region in the related art. The timing of the write operations of the servers in each region cannot be guaranteed, resulting in the disadvantage of inconsistent data in the servers in each region, improving the consistency of the data in the servers in each region, so as to realize the application of the stored data. , such as browsing the forum content in the above example, it has the technical effect of high time limit and effectiveness; and it can avoid the disadvantage that the storage stability of the multi-region storage system may decrease exponentially with the increase of the number of regions, and improve the Technical effect of storage stability of multi-regional storage.
基于上述分析，本公开实施例提供了一种多地域存储系统的数据写入方法，包括：获取写入请求，其中，写入请求中携带待写入数据，对待写入数据进行分区处理，得到待写入数据的分区区域中的分区数据，并将待写入数据的分区区域中的分区数据存储至预设队列中，控制多地域存储系统中各地域的服务器，从预设队列中读取并存储待写入数据的分区区域中的分区数据，在本实施例中，通过分区处理得到分区数据，并将分区数据存储至预设队列，以控制各地域的服务器从预设队列中获取并写入分区数据的技术特征，可以实现各地域的服务器的同步写入，避免了各地域的服务器所存储的数据不一致的弊端，提高了数据写入的可靠性和准确性，也提高了多地域存储系统的存储稳定性的技术效果。Based on the above analysis, an embodiment of the present disclosure provides a method for writing data in a multi-region storage system, including: obtaining a write request, wherein the write request carries data to be written, and partitions the data to be written to obtain Partition data in the partition area where data is to be written, and store the partition data in the partition area where data is to be written in a preset queue, control servers in each region in the multi-region storage system, and read from the preset queue and store the partition data in the partition area where the data is to be written. In this embodiment, the partition data is obtained through partition processing, and the partition data is stored in the preset queue, so as to control the servers in each region to obtain and store the partition data from the preset queue. The technical characteristics of writing partition data can realize synchronous writing of servers in various regions, avoid the disadvantage of inconsistent data stored by servers in various regions, improve the reliability and accuracy of data writing, and improve multi-regional data. Technical effects of storage stability of storage systems.
图2是根据本公开第二实施例的示意图，如图2所示，本公开实施例的多地域存储系统的数据写入方法，包括：FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure. As shown in FIG. 2 , a data writing method of a multi-region storage system according to an embodiment of the present disclosure includes:
S201：响应于接收到写入请求，确定接收到写入请求的服务器的地域类型。S201: In response to receiving the write request, determine the region type of the server that received the write request.
可以理解的是，为了避免繁琐地陈述，关于本实施例与上述实施例中相同的技术特征，本实施例不再赘述。It can be understood that, in order to avoid tedious statements, the technical features of this embodiment that are the same as those in the above-mentioned embodiments will not be repeated in this embodiment.
地域类型包括主地域类型和从地域类型，对应主地域类型的地域可以称为主地域，部署于主地域的服务器可以称为主服务器，对应从地域类型的地域可以称为从地域，部署于从地域的服务器可以称为从服务器。Region types include master region type and slave region type. The region corresponding to the master region type can be called the master region, the server deployed in the master region can be called the master server, the region corresponding to the slave region type can be called the slave region, and the region deployed in the slave region can be called the slave region. The servers in the region can be called slave servers.
其中，某一地域为主地域类型还是从地域类型，可以预先确定，如基于各地域的数据量确定主地域类型和从地域类型。示例性的，相对而言，可以将数据量较大的地域的地域类型确定为主地域类型，可以将数据量较小的地域的地域类型确定为从地域类型。Wherein, whether a certain region is the main region type or the subordinate region type can be predetermined, for example, the main region type and the subordinate region type are determined based on the data volume of each region. Exemplarily, relatively speaking, the area type of the area with a large amount of data may be determined as the main area type, and the area type of the area with a small amount of data may be determined as the subordinate area type.
例如，结合上述实施例和图3，若本实施例的方法的待写入数据为论坛中的帖子的回复内容，地域的数量为K个，如图3中所述的地域1、地域2、直至地域K。For example, in combination with the above-mentioned embodiment and FIG. 3 , if the data to be written in the method of this embodiment is the reply content of a post in the forum, and the number of regions is K, the
其中，在K个地域中，地域1中的访问论坛的用户的数量最多，发帖和回帖的内容也最多，则可以将地域1的地域类型确定为主地域类型，将K个地域中除地域1以外的其他地域的类型确定为从地域类型。即如图3所示，地域1为主地域，其他地域为从地域。Among them, among the K regions,
值得说明的是，主地域的数量可以为一个也可以为多个，上述示例只是以一个主地域为例对主地域和从地域进行示范性的说明。例如，可以从K个地域中，选择两个或者更多个访问论坛的用户的数量最多的地域为主地域。It is worth noting that the number of master regions may be one or more, and the above example is only an exemplary description of the master region and the slave regions by taking one master region as an example. For example, from the K regions, two or more regions with the largest number of users accessing the forum may be selected as the main region.
S202：若写入请求的服务器的地域类型为从地域类型，则从多地域存储系统中的各地域服务器中，确定地域类型为主地域类型的服务器。S202: If the region type of the server for which the write request is made is the slave region type, determine the region type of the server of the master region type from the region servers in the multi-region storage system.
示例性的，结合上述分析和图4，主地域的服务器和从地域的服务器均包括接入层，其中，接入层可以用于接入写入请求。写入装置可以对是否有写入请求进行监听，即写入装置可以对主地域的服务器的接入层是否有写入请求接入进行监听，也可以对从地域的服务器的接入层是否有写入请求接入进行监听，以判断写入请求的服务器的地域类型为主地域类型还是从地域类型。Exemplarily, with reference to the above analysis and FIG. 4 , both the server in the master area and the server in the slave area include an access layer, where the access layer can be used to access the write request. The writing device can monitor whether there is a write request, that is, the writing device can monitor whether the access layer of the server in the master region has access to a write request, or whether the access layer of the server in the slave region has access to the write request. The write request access is monitored to determine whether the region type of the server for the write request is the main region type or the slave region type.
一个示例中，若接入写入请求的服务器的地域类型为从地域类型，即从服务器的接入层接入了该写入请求，则确定出主服务器。In an example, if the region type of the server accessing the write request is the slave region type, that is, the access layer of the slave server has accessed the write request, the master server is determined.
另一个示例中，若接入写入请求的服务器的地域类型为主地域类型，即主服务器的接入层接入了该写入请求，则可以执行下述的分区处理的操作。In another example, if the region type of the server accessing the write request is the main region type, that is, the access layer of the main server accesses the write request, the following partition processing operation can be performed.
S203：将写入请求转发给主地域类型的服务器。S203: Forward the write request to the server of the primary domain type.
例如，结合图3，若写入请求被接入的服务器为从地域2中部署的从服务器，则可以确定出部署于主地域(即部署于地域1)的主服务器，以将接入请求转发给部署于地域1的主服务器。For example, referring to FIG. 3 , if the server to which the write request is accessed is the slave server deployed in the slave region 2, the master server deployed in the master region (ie, the region 1) can be determined to forward the access request To the master server deployed in
值得的是，在本实施例中，通过将写入请求转发给主服务器，以执行后续的写入操作，可以实现提高写入操作的准确性和可靠性的技术效果。It is worthwhile that, in this embodiment, by forwarding the write request to the master server to perform the subsequent write operation, the technical effect of improving the accuracy and reliability of the write operation can be achieved.
在一些实施例中，在确定出主服务器之后，确定主服务器为正常服务器还是异常服务器，如果主服务器为正常服务器，则可以将写入请求转发给主服务器；如果主服务器为异常服务器，则可以重新确定主服务器，从而将写入请求转发给重新确定的主服务器。In some embodiments, after the primary server is determined, it is determined whether the primary server is a normal server or an abnormal server. If the primary server is a normal server, the write request can be forwarded to the primary server; if the primary server is an abnormal server, it can be Re-primary server forwards write requests to the re-determined primary server.
其中，异常服务器是指无法正常运行的服务器，反之，正常服务器是指正常运行的服务器。The abnormal server refers to a server that cannot run normally, and vice versa, a normal server refers to a server that runs normally.
关于怎样重新确定主服务器的方法包括如下步骤：The method on how to re-determine the primary server includes the following steps:
第一步骤：若主地域类型的服务器为异常服务器，则获取地域类型为从地域类型的服务器的运行属性信息。Step 1: If the server of the master region type is an abnormal server, obtain the operation attribute information of the server of the region type of the slave region type.
第二步骤：根据获取到的运行属性信息从地域类型为从地域类型的服务器中，确定为主地域类型的服务器。Step 2: According to the obtained operation attribute information, from the servers whose region type is the slave region type, determine the server of the master region type.
示例性的，结合图3，服务器1为主地域类型的服务器，服务器2直至服务器K为从服务器，若服务器1为异常服务器，则获取服务器2直至服务器K各自对应的运行属性信息，以便从服务器2直至服务器K中确定出为主地域类型的服务器(即主服务器)。Exemplarily, with reference to FIG. 3,
其中，运行属性信息是指，与服务器性能相关的属性，如处理器利用率，运行任务数量，剩余利用资源等。The running attribute information refers to attributes related to server performance, such as processor utilization, number of running tasks, and remaining utilized resources.
在本实施例中，通过结合从地域类型的服务器的运行属性信息，确定主地域类型的服务器，可以使得确定出的主地域类型的服务器具有较高的可靠性和可依赖性的技术效果。In this embodiment, by combining the operating attribute information of the servers of the slave region type to determine the server of the master region type, the determined server of the master region type can have the technical effect of high reliability and dependability.
在一些实施例中，若根据从地域类型的服务器的运行属性信息，确定从地域类型的服务器为异常服务器，则提取待写入数据中的关键词数据，以将携带关键词数据的写入请求转发给主地域类型的服务器。In some embodiments, if it is determined that the server of the slave region type is an abnormal server according to the operation attribute information of the server of the slave region type, the keyword data in the data to be written is extracted, and the write request carrying the keyword data is extracted. Forwarded to servers of the primary domain type.
其中，关键词数据是指，用于表征待写入数据的主题内容的数据，为待写入数据的核心内容。可以通过语义分析的方式确定关键词数据。The keyword data refers to data used to represent the subject content of the data to be written, which is the core content of the data to be written. The keyword data may be determined by means of semantic analysis.
示例性的，结合上述实施例和图3，若服务器1为异常服务器，且根据服务器2直至服务器K各自对应的属性信息，确定服务器2直至服务器K均为异常服务器，则对待写入数据进行语义分析，去除待写入数据中的助词等没有实际意义的词，还可以继续去除冗余的词，得到待写入数据的关键词数据。Exemplarily, with reference to the above-mentioned embodiment and FIG. 3, if
在本实施例中，通过将关键词数据转发给主地域的服务器，可以尽可能写入待写入数据中较为重要和核心的数据，以尽可能提高写入的准确性和可靠性的技术效果。In this embodiment, by forwarding the keyword data to the server in the main region, the more important and core data in the data to be written can be written as much as possible, so as to improve the technical effect of the accuracy and reliability of writing as much as possible .
在另一些实施例中，若各服务器均为异常服务器，则可以不将从地域类型的服务器接收到写入请求转发给主地域类型的服务器，而通过从地域类型的服务器的路由层写入预设队列。且也可以是将关键词数据写入至预设队列。In other embodiments, if each server is an abnormal server, the write request received from the server of the region type may not be forwarded to the server of the main region type, and the write request may be written from the routing layer of the server of the region type. Set up a queue. Also, the keyword data may be written into the preset queue.
结合图4可知，在一些实施例中，接入层可以包括流控处理模块，用于对接入的流量进行评估。若服务器(如图4中所示的主服务器和从服务器)能够支持接入的流量，如服务器能够承担与接入的流量对应的负载等，则对接入的流量进行评估的评估结果为通过评估。It can be seen with reference to FIG. 4 that, in some embodiments, the access layer may include a flow control processing module for evaluating the access traffic. If the server (the master server and the slave server as shown in FIG. 4 ) can support the incoming traffic, for example, the server can bear the load corresponding to the incoming traffic, etc., the evaluation result of evaluating the incoming traffic is pass Evaluate.
相应的，如图4所示，接入层还可以包括权限校验模块，用于对接入的流量是否可以访问服务器进行校验。若校验通过，则接入的流量可以访问服务器，反之，服务器可以拒绝接入的流量的访问。其中，校验的方式可以通过令牌(token)和唯一标识(key)的方式实现。Correspondingly, as shown in FIG. 4 , the access layer may further include an authority verification module for verifying whether the accessed traffic can access the server. If the verification is passed, the incoming traffic can access the server, otherwise, the server can deny the access of the incoming traffic. Among them, the verification method can be realized by a token (token) and a unique identification (key).
且结合上述分析可知，接入层还可以包括流量收敛模块，用于确定接入写入请求是否为主地域类型的服务器，以便在接入写入请求的为从地域类型的服务器(如图4中的从服务器)时，将写入请求转发给为主地域类型的服务器(如图4中的主服务器)。And combined with the above analysis, it can be seen that the access layer can also include a traffic convergence module, which is used to determine whether the access write request is a server of the primary region type, so that the access write request is a server of the secondary region type (as shown in Figure 4). When the slave server in the server is used), the write request is forwarded to the server of the master domain type (the master server in Figure 4).
S204：对待写入数据进行分区处理，得到待写入数据的分区区域中的分区数据。S204: Perform partition processing on the data to be written to obtain partition data in the partition area of the data to be written.
在一些实施例中，可以基于数据类型对待写入数据进行分区处理。其中，数据类型包括字符型、宽字符型、整型、浮点型、双浮点型、布尔型等。In some embodiments, data to be written may be partitioned based on data type. Among them, the data type includes character type, wide character type, integer type, floating point type, double floating point type, Boolean type, etc.
S205：确定每一分区区域的分区数据存储至预设队列中存储顺序信息。S205: Determine that the partition data of each partition area is stored in a preset queue to store order information.
结合上述示例，分区区域的数量为N个，即有N份分区数据，则确定该N份分区数据被存储至预设队列中的顺序。With reference to the above example, if the number of partition regions is N, that is, there are N pieces of partition data, the sequence in which the N pieces of partition data are stored in the preset queue is determined.
在一些实施例中，每一分区区域具有键值；S205可以包括：计算每一分区区域的分区数据的键值的哈希值，其中，哈希值表征与哈希值对应的分区区域的分区数据在预设队列中的存储顺序信息。In some embodiments, each partition area has a key value; S205 may include: calculating a hash value of the key value of the partition data of each partition area, wherein the hash value represents the partition of the partition area corresponding to the hash value Information on the storage order of data in the preset queue.
结合上述示例，N个分区区域中的每一个分区区域有键值，各分区区域的键值分配可以基于需求、历史记录、以及试验等方式进行确定，本实施例不做限定。且本实施例对计算得到哈希值的哈希算法不做限定，如可以采用一致性哈希算法。In combination with the above examples, each of the N partitioned areas has a key value, and the key value allocation of each partitioned area may be determined based on requirements, historical records, and experiments, which are not limited in this embodiment. And this embodiment does not limit the hash algorithm for calculating the hash value, for example, a consistent hash algorithm may be used.
示例性的，结合图4和上述示例，消息队列利器(Mq-proxy)，分区区域的键值分别为key_1_op1、key_1_op2、key_2_op1，则分别进行一致性哈希算法，得到各自对应的哈希值为：partition-0、partition-1、partition-2。Exemplarily, in conjunction with FIG. 4 and the above example, the message queue tool (Mq-proxy), the key values of the partition area are respectively key_1_op1, key_1_op2, key_2_op1, then the consistent hash algorithm is performed respectively, and the corresponding hash value is obtained. : partition-0, partition-1, partition-2.
在本实施例中，通过结合哈希值表征各分区数据被存储于预设队列的存储顺序信息，可以使得存储各分区数据具有有序性，且具有较小延时，从而提高存储的有效性和可靠性的技术效果。In this embodiment, by combining the hash value to represent the storage order information of the data of each partition stored in the preset queue, the data of each partition can be stored in an orderly manner and with a small delay, thereby improving the effectiveness of storage and reliability of technical effects.
S206：根据各分区区域的分区数据各自对应的存储顺序信息，将各分区区域的分区数据存储至预设队列中。S206: Store the partition data of each partition area in a preset queue according to the corresponding storage sequence information of the partition data of each partition area.
在本实施例中，通过结合各存储顺序信息对N份分区数据进行存储，可以实现数据存储的有序性，且当后续对N份分区数据进行写入操作时，可以结合预设的先进先出的原则进行写入，从而实现写入的有效性和同步性，进而使得各地域的服务器中的数据具有较高的一致性的技术效果。In this embodiment, by storing the N pieces of partition data in combination with each storage order information, the order of data storage can be achieved, and when subsequent write operations are performed on the N pieces of partition data, it can be combined with the preset first-in-first-out Write according to the principle of writing out, so as to realize the validity and synchronization of writing, and then make the data in the servers in various regions have the technical effect of high consistency.
S207：获取各分区区域中的分区数据各自对应的数据类型。S207: Acquire respective data types corresponding to the partition data in each partition area.
示例性的，结合上述示例，确定分区区域1中分区数据的数据类型，直至确定分区区域N中分区数据的数据类型。Exemplarily, in combination with the above examples, the data type of the partition data in the
S208：根据数据类型与存储引擎之间的映射关系，控制多地域存储系统中各地域的服务器，将预设队列中的各分区区域中的分区数据写入至存储矩阵中的与各分区区域中的分区数据各自对应的存储引擎中。S208: Control the servers in each region in the multi-region storage system according to the mapping relationship between the data type and the storage engine, and write the partition data in each partition region in the preset queue into the storage matrix and in each partition region The partition data of each corresponding storage engine.
结合图4和上述示例，写入装置控制主服务器根据映射关系，从预设队列中将各分区数据写入至各分区数据各自对应的存储引擎中。且写入装置控制从服务器根据映射关系，从预设队列中将各分区数据写入至各分区数据各自对应的存储引擎中。With reference to FIG. 4 and the above example, the writing device controls the main server to write the data of each partition from the preset queue to the storage engine corresponding to the data of each partition according to the mapping relationship. And the writing device controls the slave server to write the data of each partition from the preset queue to the storage engine corresponding to the data of each partition according to the mapping relationship.
也就是说，各地域的服务器可以实现同步写入，从而实现各地域的服务器中的存储矩阵中的数据具有较高的一致性。That is to say, the servers in each region can implement synchronous writing, so that the data in the storage matrix in the servers in each region has high consistency.
示例性的，存储矩阵包括多个存储引擎，不同的存储引擎用于存储不同数据类型的数据。Exemplarily, the storage matrix includes multiple storage engines, and different storage engines are used to store data of different data types.
在一些实施例中，写入装置包括表配置，表配置可以用于表征数据类型与存储引擎之间的映射关系。因此，在对预设队列中的各分区数据执行写入操作时，可以读取表配置，以确定用于存储每一分区数据的存储引擎，并将该分区数据写入至对应的存储引擎中。In some embodiments, the writing device includes a table configuration that can be used to characterize the mapping relationship between data types and storage engines. Therefore, when the write operation is performed on the data of each partition in the preset queue, the table configuration can be read to determine the storage engine used to store the data of each partition, and the partition data can be written into the corresponding storage engine .
值得说明的是，在本实施例中，通过将待写入数据写入至包括多个存储引擎的存储矩阵中，可以实现数据存储的多元化和灵活性的技术效果。It should be noted that, in this embodiment, by writing the data to be written into a storage matrix including a plurality of storage engines, the technical effects of diversification and flexibility of data storage can be achieved.
在一些实施例中，存储矩阵可以通过对各存储引擎进行标准化处理实现。示例性的，标准化处理可以为对封装相同的应用程序接口(Application ProgrammingInterface，API)。In some embodiments, the storage matrix can be implemented by normalizing each storage engine. Exemplarily, the standardization process may be to encapsulate the same application programming interface (Application Programming Interface, API).
如为各存储引擎封装相同的应用程序接口，并基于具有相同的应用程序接口的各存储引擎构建存储矩阵。For example, the same application program interface is encapsulated for each storage engine, and a storage matrix is constructed based on each storage engine with the same application program interface.
值得说明的是，通过结合封装相同的应用程序接口构建存储矩阵，可以屏蔽各存储引擎之间的差异化，实现各存储引擎之间的解耦，且可以实现数据方便快捷地被写入至对应的存储引擎，以及方便快捷且灵活地对各存储引擎中的数据调用的技术效果。It is worth noting that by building a storage matrix by combining the same application program interface encapsulation, the differences between the storage engines can be shielded, the decoupling between the storage engines can be realized, and the data can be conveniently and quickly written to the corresponding storage engine. storage engine, and the technical effect of calling data in each storage engine conveniently, quickly and flexibly.
存储矩阵中至少包括全文搜索和分析引擎(Elasticsearch，ES)(如图4中的ES存储引擎)、以及查看表类型存储引擎(如图4中的table存储引擎)等，也可以包括不同的存储引擎组合而得到的新的存储引擎，如图4中的“ES+table存储引擎”。The storage matrix includes at least a full-text search and analysis engine (Elasticsearch, ES) (such as the ES storage engine in Figure 4), and a viewing table type storage engine (such as the table storage engine in Figure 4), etc., and can also include different storage The new storage engine obtained by combining the engines, such as the "ES+table storage engine" in Figure 4.
例如，可以将部分分区数据存储于ES存储引擎，以便于当接收到检索服务请求时，基于ES存储引擎实现。For example, some partition data can be stored in the ES storage engine, so that when a retrieval service request is received, it can be implemented based on the ES storage engine.
可以将部分分区数据存储于table存储引擎，以便当接收到数据读取的服务请求时，基于table存储引擎实现。Part of the partitioned data can be stored in the table storage engine, so that when a service request for data reading is received, it can be implemented based on the table storage engine.
应该理解的是，上述示例只是用于示范性的说明，本实施例可能构建存储矩阵的方式，以及存储矩阵中可能包括的存储引擎，而不能理解为对存储矩阵的限定。It should be understood that the above examples are only used for exemplary illustration, and the manner in which the storage matrix may be constructed in this embodiment, and the storage engine that may be included in the storage matrix, should not be construed as a limitation on the storage matrix.
在一些实施例中，根据数据类型与存储引擎之间的映射关系，将预设队列中的各分区区域中的分区数据写入至存储矩阵中的与各分区区域中的分区数据各自对应的存储引擎中，包括如下步骤：In some embodiments, according to the mapping relationship between the data type and the storage engine, the partition data in each partition area in the preset queue is written to the storage matrix corresponding to the partition data in each partition area. The engine includes the following steps:
第一步骤：根据各分区区域中的分区数据各自对应的存储顺序信息，依次从预设队列中读取分区区域中的分区数据。The first step is to sequentially read the partition data in the partition area from the preset queue according to the corresponding storage sequence information of the partition data in each partition area.
示例性的，基于“先进先出”规则，对预设队列中的各分区数据进行读取。Exemplarily, based on the "first-in, first-out" rule, the data of each partition in the preset queue is read.
第二步骤：根据映射关系，确定与读取到的分区区域中的分区数据对应的存储引擎。Step 2: According to the mapping relationship, determine the storage engine corresponding to the read partition data in the partition area.
结合上述分析，每一分区数据具有数据类型，映射关系用于表征数据类型与存储引擎之间的关系，即针对任意存储引擎，映射关系表征该存储引擎用于存储的数据的数据类型。Combined with the above analysis, each partition data has a data type, and the mapping relationship is used to represent the relationship between the data type and the storage engine, that is, for any storage engine, the mapping relationship represents the data type of the data stored by the storage engine.
第三步骤：将读取到的分区区域中的分区数据写入至存储矩阵中的与读取到的分区区域中的分区数据对应的存储引擎中。Step 3: Write the read partition data in the partition area into a storage engine in the storage matrix corresponding to the read partition data in the partition area.
示例性地，针对读取到的每一分区数据，可以基于映射关系确定用于存储该分区数据的存储引擎，因此，可以将该分区数据写入至用于存储该分区数据的存储引擎。Exemplarily, for each read partition data, a storage engine for storing the partition data may be determined based on the mapping relationship, and therefore, the partition data may be written to the storage engine for storing the partition data.
在本实施例中，通过结合存储顺序信息读取并写入分区数据，且结合映射关系实现分区数据的写入，可以实现各地域的服务器的数据写入的一致性和同一性的技术效果。In this embodiment, by reading and writing partition data in combination with storage sequence information, and implementing partition data writing in combination with a mapping relationship, the technical effect of consistency and identity of data writing of servers in different regions can be achieved.
在一些实施例中，可以结合现场锁机制实现分区数据的写入。如获取现场锁，基于获取到的现场锁对分区数据进行写入，并在写入完成后，示范现场锁，以避免重复写入等问题，从而提高数据写入的可靠性和有效性的技术效果。In some embodiments, the writing of partitioned data may be implemented in conjunction with a field lock mechanism. For example, acquiring the field lock, writing the partition data based on the acquired field lock, and demonstrating the field lock after the writing is completed to avoid problems such as repeated writing, thereby improving the reliability and effectiveness of data writing. Effect.
S209：若任意存储矩阵中存储引擎被写入分区区域中的分区数据时写入失败，则对预设队列中的待写入数据的各分区区域中的分区数据进行异步回滚处理。S209 : If the storage engine in any storage matrix fails to write the partition data in the partition area, perform asynchronous rollback processing on the partition data in each partition area where the data to be written in the preset queue is to be written.
示例性的，结合上述分析，若在写入某份分区数据时，写入失败，则重新写入待写入数据，即从N份分区数据中的第一份分区数据重新对待写入数据进行写入处理。Exemplarily, in combination with the above analysis, if the writing fails when writing a certain partition data, the data to be written is rewritten, that is, the data to be written is rewritten from the first partition data in the N partition data. Write processing.
且在本实施例中，回滚处理为异步方式的回滚处理，异步方式的回滚处理可以理解为不影响其他数据的写入的回滚处理。And in this embodiment, the rollback processing is asynchronous rollback processing, and the asynchronous rollback processing can be understood as rollback processing that does not affect the writing of other data.
如执行异步回滚处理时，还是可以继续写入其他的数据，以并行的方式实现两个操作，从而实现灵活性的技术效果。For example, when performing asynchronous rollback processing, it is still possible to continue to write other data and implement two operations in a parallel manner, thereby achieving the technical effect of flexibility.
图5是根据本公开第三实施例的示意图，如图5所示，本公开实施例的多地域存储系统的数据写入装置500，包括：FIG. 5 is a schematic diagram according to a third embodiment of the present disclosure. As shown in FIG. 5 , a
获取单元501，用于获取写入请求，其中，写入请求中携带待写入数据。The obtaining
分区单元502，用于对待写入数据进行分区处理，得到待写入数据的分区区域中的分区数据。The
存储单元503，用于将待写入数据的分区区域中的分区数据存储至预设队列中。The
控制单元504，用于控制多地域存储系统中各地域的服务器，从预设队列中读取并存储待写入数据的分区区域中的分区数据。The
图6是根据本公开第四实施例的示意图，如图6所示，本公开实施例的多地域存储系统的数据写入装置600，包括：FIG. 6 is a schematic diagram according to a fourth embodiment of the present disclosure. As shown in FIG. 6 , a
获取单元601，用于获取写入请求，其中，写入请求中携带待写入数据。The obtaining unit 601 is configured to obtain a write request, wherein the write request carries the data to be written.
结合图6可知，在一些实施例中，获取单元601，包括：6 , in some embodiments, the obtaining unit 601 includes:
第二确定子单元6011，用于响应于接收到写入请求，确定接收到写入请求的服务器的地域类型。The second determining
第三确定子单元6012，用于若写入请求的服务器的地域类型为从地域类型，则从多地域存储系统中的各地域服务器中，确定地域类型为主地域类型的服务器。The third determining
转发子单元6013，用于将写入请求转发给主地域类型的服务器。The forwarding
结合图6可知，在一些实施例中，获取单元601，还包括：6 , in some embodiments, the obtaining unit 601 further includes:
第二获取子单元6014，用于若主地域类型的服务器为异常服务器，则获取地域类型为从地域类型的服务器的运行属性信息。The second obtaining
第四确定子单元6015，用于根据获取到的运行属性信息从地域类型为从地域类型的服务器中，确定为主地域类型的服务器。The
结合图6可知，在一些实施例中，获取单元601，还包括：6 , in some embodiments, the obtaining unit 601 further includes:
提取子单元6016，用于若根据从地域类型的服务器的运行属性信息，确定从地域类型的服务器为异常服务器，则提取待写入数据中的关键词数据。The
相应的，转发子单元6013用于，将携带关键词数据的写入请求转发给主地域类型的服务器。Correspondingly, the forwarding
分区单元602，用于对待写入数据进行分区处理，得到待写入数据的分区区域中的分区数据。The
存储单元603，用于将待写入数据的分区区域中的分区数据存储至预设队列中。The
在一些实施例中，待写入数据的分区区域的数量为多个；结合图6可知，存储单元603，包括：In some embodiments, the number of partition areas to which data is to be written is multiple; it can be seen from FIG. 6 that the
第一确定子单元6031，用于确定每一分区区域的分区数据存储至预设队列中存储顺序信息。The first determining
在一些实施例中，每一分区区域具有键值；第一确定子单元用于，计算每一分区区域的分区数据的键值的哈希值，其中，哈希值表征与哈希值对应的分区区域的分区数据在预设队列中的存储顺序信息。In some embodiments, each partition area has a key value; the first determination subunit is used to calculate a hash value of the key value of the partition data of each partition area, wherein the hash value represents the hash value corresponding to the hash value Information on the storage order of the partition data in the partition area in the preset queue.
存储子单元6032，用于根据各分区区域的分区数据各自对应的存储顺序信息，将各分区区域的分区数据存储至预设队列中。The
控制单元604，用于控制多地域存储系统中各地域的服务器，从预设队列中读取并存储待写入数据的分区区域中的分区数据。The
在一些实施例中，待写入数据的分区区域的数量为多个；结合图6可知，控制单元604，包括：In some embodiments, the number of partition areas to which data is to be written is multiple; it can be seen from FIG. 6 that the
第一获取子单元6041，用于获取各分区区域中的分区数据各自对应的数据类型。The first obtaining
写入子单元6042，用于根据数据类型与存储引擎之间的映射关系，将预设队列中的各分区区域中的分区数据写入至存储矩阵中的与各分区区域中的分区数据各自对应的存储引擎中。The
在一些实施例中，每一分区区域的分数数据具有存储至所述预设队列的存储顺序信息；写入子单元6042，包括：In some embodiments, the score data of each partition area has storage order information stored in the preset queue; the
读取模块用于，根据各分区区域中的分区数据各自对应的存储顺序信息，依次从预设队列中读取分区区域中的分区数据。The reading module is used for sequentially reading the partition data in the partition area from the preset queue according to the corresponding storage sequence information of the partition data in each partition area.
确定模块，用于根据映射关系，确定与读取到的分区区域中的分区数据对应的存储引擎。The determining module is configured to determine the storage engine corresponding to the read partition data in the partition area according to the mapping relationship.
写入模块，用于将读取到的分区区域中的分区数据写入至存储矩阵中的与读取到的分区区域中的分区数据对应的存储引擎中。The writing module is configured to write the read partition data in the partition area into the storage engine in the storage matrix corresponding to the read partition data in the partition area.
结合图6可知，在一些实施例中，控制单元604，还包括：6 , in some embodiments, the
封装子单元6043，用于为各存储引擎封装相同的应用程序接口。The
构建子单元6044，用于基于具有相同的应用程序接口的各存储引擎构建存储矩阵。A
结合图6可知，在一些实施例中，控制单元604，还包括：6 , in some embodiments, the
回滚子单元6045，用于若任意存储矩阵中存储引擎被写入分区区域中的分区数据时写入失败，则对预设队列中的待写入数据的各分区区域中的分区数据进行异步回滚处理。The
根据本公开实施例的另一个方面，本公开实施例还提供了一种多地域存储系统，多地域存储系统通过如上任一实施例所述的方法存储待写入数据，多地域存储系统包括：各地域各自对应的服务器，每一地域的服务器包括存储矩阵，存储矩阵包括多个存储引擎。According to another aspect of the embodiments of the present disclosure, the embodiments of the present disclosure further provide a multi-regional storage system, where the multi-regional storage system stores data to be written by using the method described in any of the above embodiments, and the multi-regional storage system includes: Each region corresponds to a server, the server in each region includes a storage matrix, and the storage matrix includes multiple storage engines.
在一些实施例中，存储矩阵是为各存储引擎封装相同的应用程序接口，并基于具有相同的应用程序接口的各存储引擎构建的。In some embodiments, the storage matrix encapsulates the same API for each storage engine and is built on top of each storage engine with the same API.
示例性的，当多地域存储系统中包括两个地域的服务器时，其中一个地域的服务器可以为主服务器，一个地域的服务器可以为从服务器，且结构示意图可以参阅图4，相关描述可以参见上述实施例，此处不再赘述。Exemplarily, when the multi-regional storage system includes servers in two regions, the server in one region can be the master server, and the server in the other region can be the slave server, and the schematic diagram can refer to FIG. 4 , and the related description can refer to the above. Examples are not repeated here.
根据本公开实施例的另一个方面，本公开实施例还提供了一种多地域存储系统的数据写入系统，包括：According to another aspect of the embodiments of the present disclosure, the embodiments of the present disclosure further provide a data writing system for a multi-region storage system, including:
如上任一实施例所述的装置；The apparatus of any of the above embodiments;
如上任一实施例所述的多地域存储系统。The multi-region storage system as described in any of the above embodiments.
图7是根据本公开第五实施例的示意图，如图7所示，本公开中的电子设备700可以包括：处理器701和存储器702。FIG. 7 is a schematic diagram according to a fifth embodiment of the present disclosure. As shown in FIG. 7 , an
存储器702，用于存储程序；存储器702，可以包括易失性存储器(英文：volatilememory)，例如随机存取存储器(英文：random-access memory，缩写：RAM)，如静态随机存取存储器(英文：static random-access memory，缩写：SRAM)，双倍数据率同步动态随机存取存储器(英文：Double Data Rate Synchronous Dynamic Random Access Memory，缩写：DDR SDRAM)等；存储器也可以包括非易失性存储器(英文：non-volatile memory)，例如快闪存储器(英文：flash memory)。存储器702用于存储计算机程序(如实现上述方法的应用程序、功能模块等)、计算机指令等，上述的计算机程序、计算机指令等可以分区存储在一个或多个存储器702中。并且上述的计算机程序、计算机指令、数据等可以被处理器701调用。The
上述的计算机程序、计算机指令等可以分区存储在一个或多个存储器702中。并且上述的计算机程序、计算机指据等可以被处理器701调用。The computer programs, computer instructions, etc. described above may be partitioned and stored in one or more of the
处理器701，用于执行存储器702存储的计算机程序，以实现上述实施例涉及的方法中的各个步骤。The
具体可以参见前面方法实施例中的相关描述。For details, refer to the relevant descriptions in the foregoing method embodiments.
处理器701和存储器702可以是独立结构，也可以是集成在一起的集成结构。当处理器701和存储器702是独立结构时，存储器702、处理器701可以通过总线703耦合连接。The
本实施例的电子设备可以执行上述方法中的技术方案，其具体实现过程和技术原理相同，此处不再赘述。The electronic device in this embodiment can execute the technical solutions in the foregoing methods, and the specific implementation process and technical principles thereof are the same, and are not repeated here.
本公开的技术方案中，所涉及的用户个人信息(如待写入数据)的收集、存储、使用、加工、传输、提供和公开等处理，均符合相关法律法规的规定，且不违背公序良俗。In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of the involved user personal information (such as data to be written) are in compliance with relevant laws and regulations, and do not violate public order and good customs.
根据本公开的实施例，本公开还提供了一种电子设备、一种可读存储介质和一种计算机程序产品。According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
根据本公开的实施例，本公开还提供了一种计算机程序产品，计算机程序产品包括：计算机程序，计算机程序存储在可读存储介质中，电子设备的至少一个处理器可以从可读存储介质读取计算机程序，至少一个处理器执行计算机程序使得电子设备执行上述任一实施例提供的方案。According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, the computer program product includes: a computer program, the computer program is stored in a readable storage medium, and at least one processor of the electronic device can read from the readable storage medium A computer program is taken, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the foregoing embodiments.
图8示出了可以用来实施本公开的实施例的示例电子设备800的示意性框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字助理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。FIG. 8 shows a schematic block diagram of an example
如图8所示，设备800包括计算单元801，其可以根据存储在只读存储器(ROM)802中的计算机程序或者从存储单元808加载到随机访问存储器(RAM)803中的计算机程序，来执行各种适当的动作和处理。在RAM 803中，还可存储设备800操作所需的各种程序和数据。计算单元801、ROM 802以及RAM 803通过总线804彼此相连。输入/输出(I/O)接口805也连接至总线804。As shown in FIG. 8 , the
设备800中的多个部件连接至I/O接口805，包括：输入单元806，例如键盘、鼠标等；输出单元807，例如各种类型的显示器、扬声器等；存储单元808，例如磁盘、光盘等；以及通信单元809，例如网卡、调制解调器、无线通信收发机等。通信单元809允许设备800通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。Various components in the
计算单元801可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元801的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元801执行上文所描述的各个方法和处理，例如多地域存储系统的数据写入方法。例如，在一些实施例中，多地域存储系统的数据写入方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元808。在一些实施例中，计算机程序的部分或者全部可以经由ROM 802和/或通信单元809而被载入和/或安装到设备800上。当计算机程序加载到RAM 803并由计算单元801执行时，可以执行上文描述的多地域存储系统的数据写入方法的一个或多个步骤。备选地，在其他实施例中，计算单元801可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行多地域存储系统的数据写入方法。
本文中以上描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、复杂可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。Various implementations of the systems and techniques described herein above may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), complex programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, performs the functions/functions specified in the flowcharts and/or block diagrams. Action is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以解决了传统物理主机与VPS服务("Virtual Private Server"，或简称"VPS")中，存在的管理难度大，业务扩展性弱的缺陷。服务器也可以为分布式系统的服务器，或者是结合了区块链的服务器。A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or a cloud host. It is a host product in the cloud computing service system to solve the traditional physical host and VPS service ("Virtual Private Server", or "VPS" for short). , there are the defects of difficult management and weak business expansion. The server can also be a server of a distributed system, or a server combined with a blockchain.
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本发公开中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders. As long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, there is no limitation herein.
上述具体实施方式，并不构成对本公开保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本公开的精神和原则之内所作的修改、等同替换和改进等，均应包含在本公开保护范围之内。The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.
Claims (26)
Translated fromChinesePriority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210110211.5ACN114442959B (en) | 2022-01-29 | 2022-01-29 | Data writing method, device and system of multi-region storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210110211.5ACN114442959B (en) | 2022-01-29 | 2022-01-29 | Data writing method, device and system of multi-region storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114442959Atrue CN114442959A (en) | 2022-05-06 |
| CN114442959B CN114442959B (en) | 2024-08-13 |
Family
ID=81371688
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210110211.5AActiveCN114442959B (en) | 2022-01-29 | 2022-01-29 | Data writing method, device and system of multi-region storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114442959B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140244949A1 (en)* | 2013-02-26 | 2014-08-28 | International Business Machines Corporation | Asynchronous Data Mirroring in Memory Controller |
| US20160188426A1 (en)* | 2014-12-31 | 2016-06-30 | International Business Machines Corporation | Scalable distributed data store |
| CN111343277A (en)* | 2020-03-04 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Distributed data storage method, system, computer device and storage medium |
| CN112115113A (en)* | 2020-09-25 | 2020-12-22 | 北京百度网讯科技有限公司 | Data storage system, method, apparatus, device, and storage medium |
| CN112527899A (en)* | 2020-12-03 | 2021-03-19 | 北京百度网讯科技有限公司 | Data synchronization method, device, equipment and storage medium |
| CN113407637A (en)* | 2021-07-13 | 2021-09-17 | 上海浦东发展银行股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
- 2022
- 2022-01-29CNCN202210110211.5Apatent/CN114442959B/enactiveActive
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140244949A1 (en)* | 2013-02-26 | 2014-08-28 | International Business Machines Corporation | Asynchronous Data Mirroring in Memory Controller |
| US20160188426A1 (en)* | 2014-12-31 | 2016-06-30 | International Business Machines Corporation | Scalable distributed data store |
| CN111343277A (en)* | 2020-03-04 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Distributed data storage method, system, computer device and storage medium |
| CN112115113A (en)* | 2020-09-25 | 2020-12-22 | 北京百度网讯科技有限公司 | Data storage system, method, apparatus, device, and storage medium |
| CN112527899A (en)* | 2020-12-03 | 2021-03-19 | 北京百度网讯科技有限公司 | Data synchronization method, device, equipment and storage medium |
| CN113407637A (en)* | 2021-07-13 | 2021-09-17 | 上海浦东发展银行股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| TAMER Z. EMARA等: "Distributed Data Strategies to Support Large-Scale Data Analysis Across Geo-Distributed Data Centers", IEEE ACCESS, 29 September 2020 (2020-09-29)* |
| 李丹;叶廷东;: ""异地多活"分布式存储系统设计和实现", 计算机测量与控制, no. 04, 25 April 2020 (2020-04-25)* |
| 蔡斌;谢长生;朱光喜;: "基于分散式体系结构的高可靠文件存储系统的研究", 计算机科学, no. 09, 25 September 2008 (2008-09-25)* |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114442959B (en) | 2024-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11947842B2 (en) | Method for writing data in append mode, device and storage medium | |
| CN113961510B (en) | File processing method, device, equipment and storage medium | |
| CN112970011B (en) | Pedigree in record query optimization | |
| US20250086067A1 (en) | Method and apparatus for fast generating a full snapshot, electronic device and storage medium | |
| US20230048813A1 (en) | Method of storing data and method of reading data | |
| US20230012642A1 (en) | Method and device for snapshotting metadata, and storage medium | |
| CN112860811A (en) | Method and device for determining data blood relationship, electronic equipment and storage medium | |
| CN112528067A (en) | Graph database storage method, graph database reading method, graph database storage device, graph database reading device and graph database reading equipment | |
| CN114035750A (en) | File processing method, device, equipment, medium and product | |
| CN115639966A (en) | Data writing method and device, terminal equipment and storage medium | |
| US20220107949A1 (en) | Method of optimizing search system | |
| CN115328898A (en) | A data processing method, device, electronic device and medium | |
| CN114442959B (en) | Data writing method, device and system of multi-region storage system | |
| WO2021189283A1 (en) | Method and device for data processing, electronic device, and storage medium | |
| CN115203489A (en) | Dynamic graph data storage system, reading system and corresponding method | |
| CN114996243A (en) | Database operation method and device, electronic equipment and storage medium | |
| CN112084141B (en) | A method, device, equipment and medium for expanding full-text search system | |
| CN114816790A (en) | Data processing method, device, device and medium for dicing process | |
| CN114281880A (en) | Capacity expansion method and device, electronic equipment and computer readable medium | |
| US12353721B2 (en) | Method, electronic device, and computer program product for processing key-value data | |
| CN119336841B (en) | Data synchronization method, device, electronic device and storage medium | |
| CN116700824B (en) | Configuration strategy synchronization method, device and computer program product | |
| US20250217316A1 (en) | Attribute information acquisition method and apparatus for directory of distributed file system,and device | |
| US12326866B1 (en) | Out-of-core BFS for shortest path graph queries | |
| CN119782261B (en) | Method, device, equipment and storage medium for updating attribute of directory of distributed file system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |