技术领域technical field
本公开涉及分布式文件处理技术领域，尤其涉及应用于分布式文件系统的日志重演方法、装置及终端。The present disclosure relates to the technical field of distributed file processing, and in particular to a log playback method, device and terminal applied to a distributed file system.
背景技术Background technique
分布式文件系统一般通过事务机制来保证文件系统元数据的一致性。该机制通常使用journal的方式实现，即对文件系统的元数据的修改操作，先记录到一个编辑日志(edit log)里头，然后再进行实际(in-place)的元数据修改操作。这样可以保证任何一个对文件系统的元数据的修改要么全部完成要么什么都不做，从而就保证了文件系统元数据之间的一致性。在系统宕机，例如掉电、软件错误等等，之后后重启的过程中，编辑日志中存在中针对各中文件系统操作的记录，需要对那些存在于编辑日志中但并没有做或者只做了一部分实际修改的记录重新进行in-place更新(re-do log)，这个过程叫做文件系统的恢复，即日志重演(log replay)。Distributed file systems generally use transaction mechanisms to ensure the consistency of file system metadata. This mechanism is usually implemented in a journal manner, that is, the modification operation of the metadata of the file system is first recorded in an edit log (edit log), and then the actual (in-place) metadata modification operation is performed. In this way, it can be guaranteed that any modification to the metadata of the file system is either completely completed or nothing is done, thereby ensuring the consistency among the metadata of the file system. In the process of system downtime, such as power failure, software error, etc., and then restarting, there are records in the edit log for various file system operations, and those that exist in the edit log but have not been done or only done Part of the actually modified records are re-updated in-place (re-do log). This process is called file system recovery, that is, log replay.
相关技术中，在编辑日志中各记录之间往往有依赖关系。比如编辑日志中具有有这样四条记录:(1)创建文件/tmp/file1；(2)为文件/user/xyz/file分配一个数据块；(3)修改文件/tmp/file1的读写权限；(4)设置文件/user/xyz/file的长度和最后修改时间。这四个记录中,记录3依赖于记录1，记录4依赖于记录2，就是说记录3记录4必须等到记录1记录2的in-place更新完成之后才能做它们自身的in-place更新。从而现有技术中，都采用单线程的方式来完成日志重演，即采用单线程针对各项记录依次进行in-place更新，完成日志重演。In related technologies, there is often a dependency relationship between records in an edit log. For example, there are four records in the edit log: (1) create file /tmp/file1; (2) allocate a data block for file /user/xyz/file; (3) modify the read and write permissions of file /tmp/file1; (4) Set the length and last modification time of the file /user/xyz/file. Among these four records, record 3 depends on record 1, and record 4 depends on record 2, that is to say, record 3 and record 4 must wait for the in-place update of record 1 and record 2 to complete their own in-place update. Therefore, in the prior art, log replay is completed in a single-threaded manner, that is, a single thread is used to sequentially perform in-place updates for each record to complete log replay.
然而相关技术中，单线程对各项记录依次进行in-place更新，完成日志重演的方式，由于是一项项的一次对各记录进行in-place更新，会导致日志重演的速度很慢，日志重演的过程较长，会导致分布式文件系统不可用的时间变长。However, in related technologies, a single thread performs in-place updates on each record in turn to complete the log replay method. Since the in-place update is performed on each record one by one, the speed of log replay is very slow. The replay process is longer, which will cause the distributed file system to be unavailable for a longer period of time.
发明内容Contents of the invention
为克服相关技术中存在的问题，本公开提供一种应用于分布式文件系统的日志重演方法、装置及终端，用于解决现有技术中单线程对各项记录依次进行in-place更新，完成日志重演的方式，由于是一项项的一次对各记录进行in-place更新，会导致日志重演的速度很慢，日志重演的过程较长，会导致分布式文件系统不可用的时间变长的问题。In order to overcome the problems existing in the related technologies, the present disclosure provides a log replay method, device and terminal applied to a distributed file system, which is used to solve the in-place update of each record sequentially by a single thread in the prior art, and complete The way of log replay, because it is an in-place update of each record one by one, will cause the speed of log replay to be very slow, and the process of log replay will be longer, which will cause the distributed file system to be unavailable for a longer time question.
根据本公开实施例的第一方面，提供一种应用于分布式文件系统的日志重演方法，包括：According to the first aspect of the embodiments of the present disclosure, there is provided a log replay method applied to a distributed file system, including:
根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；According to the file identification of each record in the editing log, each record is divided into at least one type of record, wherein each type of record is a record with a dependency relationship;
根据划分出的记录的类数，确定用于日志重演的线程数量；Determine the number of threads used for log replay according to the number of recorded records;
采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。Each thread of the determined number of threads is used to perform log replay for various records corresponding to each thread one by one.
进一步地，所述根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，包括：Further, according to the file identification of each record in the editing log, dividing each record into at least one type of record includes:
根据所述编辑日志中的各记录，确定所述编辑日志中的各同步点；determining each synchronization point in the edit log based on each record in the edit log;
将相邻的同步点之间的各记录，确定为同时处理的各记录；Determining each record between adjacent synchronization points as each record processed at the same time;
根据确定为同时处理的各记录的文件标识，将确定为同时处理的各记录分为至少一类记录。The records determined to be processed simultaneously are classified into at least one type of records according to the file identifiers of the records determined to be processed simultaneously.
进一步地，所述根据划分出的记录的类数，确定用于日志重演的线程数量，包括：Further, the determining the number of threads used for log replay according to the class number of the divided records includes:
对各相邻的同步点之间的各类记录，确定各相邻的同步点之间的线程数量。For all types of records between adjacent synchronization points, determine the number of threads between adjacent synchronization points.
进一步地，所述采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演，包括：Further, the said use of each thread with the determined number of threads to perform log replay on various records corresponding to each thread one-to-one includes:
重复以下过程，直至对所述编辑日志中的所有记录完成日志重演：Repeat the following process until log replay is complete for all records in the edit log:
采用相邻的同步点之间的线程数量的线程，对于各线程一一对应的各类记录进行日志重演；Use the number of threads between adjacent synchronization points to perform log replay for all kinds of records corresponding to each thread;
对同步点进行日志重演。Log replay for syncpoints.
进一步地，所述同步点为所述编辑日志中的rename操作。Further, the synchronization point is a rename operation in the edit log.
进一步地，所述文件标识为文件路径前缀。Further, the file identifier is a file path prefix.
本公开的实施例提供的技术方案可以包括以下有益效果：通过根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；根据划分出的记录的类数，确定用于日志重演的线程数量；采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。从而可以依据编辑日志中的各记录的文件路径前缀，对各记录进行分类，可以保证每一类记录中的各记录的依赖关系；进而可以采用多个线程并发的重演每个记录类别里头的纪录，加快日志重演的速度，可以保证缩短分布式文件系统不可用的时间；并且，编辑日志中的各记录中的时序关系也没有被打破。The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: according to the file identification of each record in the editing log, each record is divided into at least one type of record, wherein each type of record is a record with a dependency relationship; according to The number of categories of records is divided to determine the number of threads used for log replay; each thread of the determined number of threads is used to perform log replay for each type of record corresponding to each thread one by one. Therefore, each record can be classified according to the file path prefix of each record in the edit log, and the dependency relationship of each record in each type of record can be guaranteed; then multiple threads can be used to replay the records in each record category concurrently , speeding up the log replay speed can ensure that the unavailable time of the distributed file system is shortened; moreover, the timing relationship among the records in the edit log is not broken.
根据本公开实施例的第二方面，提供一种一种应用于分布式文件系统的日志重演装置，包括：According to the second aspect of the embodiments of the present disclosure, there is provided a log playback device applied to a distributed file system, including:
分类模块，被配置为根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；The classification module is configured to classify each record into at least one type of record according to the file identifier of each record in the editing log, wherein each type of record is a record with a dependency relationship;
确定模块，被配置为根据划分出的记录的类数，确定用于日志重演的线程数量；The determination module is configured to determine the number of threads used for log replay according to the number of divided records;
重演模块，被配置为采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。The playback module is configured to use the determined number of threads to perform log playback on various records corresponding to each thread one by one.
进一步地，所述分类模块，被具体配置为：Further, the classification module is specifically configured as:
根据所述编辑日志中的各记录，确定所述编辑日志中的各同步点；determining each synchronization point in the edit log based on each record in the edit log;
将相邻的同步点之间的各记录，确定为同时处理的各记录；Determining each record between adjacent synchronization points as each record processed at the same time;
根据确定为同时处理的各记录的文件标识，将确定为同时处理的各记录分为至少一类记录。The records determined to be processed simultaneously are classified into at least one type of records according to the file identifiers of the records determined to be processed simultaneously.
进一步地，所述确定模块，被具体配置为：Further, the determination module is specifically configured as:
对各相邻的同步点之间的各类记录，确定各相邻的同步点之间的线程数量。For all types of records between adjacent synchronization points, determine the number of threads between adjacent synchronization points.
进一步地，所述重演模块，被具体配置为：Further, the playback module is specifically configured as:
重复以下过程，直至对所述编辑日志中的所有记录完成日志重演：Repeat the following process until log replay is complete for all records in the edit log:
采用相邻的同步点之间的线程数量的线程，对于各线程一一对应的各类记录进行日志重演；Use the number of threads between adjacent synchronization points to perform log replay for all kinds of records corresponding to each thread;
对同步点进行日志重演。Log replay for syncpoints.
进一步地，所述同步点为所述编辑日志中的rename操作。Further, the synchronization point is a rename operation in the edit log.
进一步地，所述文件标识为文件路径前缀。Further, the file identifier is a file path prefix.
本公开的实施例提供的技术方案可以包括以下有益效果：通过根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；根据划分出的记录的类数，确定用于日志重演的线程数量；采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。从而可以依据编辑日志中的各记录的文件路径前缀，对各记录进行分类，可以保证每一类记录中的各记录的依赖关系；进而可以采用多个线程并发的重演每个记录类别里头的纪录，加快日志重演的速度，可以保证缩短分布式文件系统不可用的时间；并且，编辑日志中的各记录中的时序关系也没有被打破。The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: according to the file identification of each record in the editing log, each record is divided into at least one type of record, wherein each type of record is a record with a dependency relationship; according to The number of categories of records is divided to determine the number of threads used for log replay; each thread of the determined number of threads is used to perform log replay for each type of record corresponding to each thread one by one. Therefore, each record can be classified according to the file path prefix of each record in the edit log, and the dependency relationship of each record in each type of record can be guaranteed; then multiple threads can be used to replay the records in each record category concurrently , speeding up the log replay speed can ensure that the unavailable time of the distributed file system is shortened; moreover, the timing relationship among the records in the edit log is not broken.
根据本公开实施例的第三方面，提供一种终端，包括：According to a third aspect of the embodiments of the present disclosure, a terminal is provided, including:
处理器，以及用于存储处理器可执行指令的存储器；a processor, and memory for storing instructions executable by the processor;
其中，所述处理器被配置为：根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；根据划分出的记录的类数，确定用于日志重演的线程数量；采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。Wherein, the processor is configured to: divide each record into at least one type of record according to the file identifier of each record in the editing log, wherein each type of record is a record with a dependency relationship; Number, determine the number of threads used for log replay; use each thread of the determined thread number to perform log replay for various records corresponding to each thread one by one.
本公开的实施例提供的技术方案可以包括以下有益效果：通过根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；根据划分出的记录的类数，确定用于日志重演的线程数量；采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。从而可以依据编辑日志中的各记录的文件路径前缀，对各记录进行分类，可以保证每一类记录中的各记录的依赖关系；进而可以采用多个线程并发的重演每个记录类别里头的纪录，加快日志重演的速度，可以保证缩短分布式文件系统不可用的时间；并且，编辑日志中的各记录中的时序关系也没有被打破。The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: according to the file identification of each record in the editing log, each record is divided into at least one type of record, wherein each type of record is a record with a dependency relationship; according to The number of categories of records is divided to determine the number of threads used for log replay; each thread of the determined number of threads is used to perform log replay for each type of record corresponding to each thread one by one. Therefore, each record can be classified according to the file path prefix of each record in the edit log, and the dependency relationship of each record in each type of record can be guaranteed; then multiple threads can be used to replay the records in each record category concurrently , speeding up the log replay speed can ensure that the unavailable time of the distributed file system is shortened; moreover, the timing relationship among the records in the edit log is not broken.
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.
附图说明Description of drawings
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本发明的实施例，并与说明书一起用于解释本发明的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention.
图1是根据一示例性实施例示出的一种应用于分布式文件系统的日志重演方法实施例一的流程图；Fig. 1 is a flow chart of Embodiment 1 of a log replay method applied to a distributed file system according to an exemplary embodiment;
图2是根据一示例性实施例示出的一种应用于分布式文件系统的日志重演方法实施例二的流程图；Fig. 2 is a flow chart of Embodiment 2 of a log replay method applied to a distributed file system according to an exemplary embodiment;
图3是根据一示例性实施例示出的一种应用推荐装置实施例三的框图；Fig. 3 is a block diagram of a third embodiment of an application recommendation device according to an exemplary embodiment;
图4是根据一示例性实施例示出的一种终端的实体的框图；Fig. 4 is a block diagram showing entities of a terminal according to an exemplary embodiment;
图5是根据一示例性实施例示出的一种终端设备800的框图。Fig. 5 is a block diagram showing a terminal device 800 according to an exemplary embodiment.
具体实施方式detailed description
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本发明相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本发明的一些方面相一致的装置和方法的例子。Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with aspects of the invention as recited in the appended claims.
图1是根据一示例性实施例示出的一种应用于分布式文件系统的日志重演方法实施例一的流程图，如图1所示，应用于分布式文件系统的日志重演方法用于终端中，该方法包括以下步骤。Fig. 1 is a flow chart of Embodiment 1 of a log replay method applied to a distributed file system according to an exemplary embodiment. As shown in Fig. 1 , the log replay method applied to a distributed file system is used in a terminal , the method includes the following steps.
在步骤S11中，根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录。In step S11, according to the file identification of each record in the editing log, each record is divided into at least one type of record, wherein each type of record is a record with a dependency relationship.
其中，所述文件标识为文件路径前缀。Wherein, the file identifier is a file path prefix.
在本步骤中，确定编辑日志中各记录的文件标识，该文件标识可以采用文件路径前缀，从而可以依据编辑日志中各记录的文件标识，将各记录分为多类记录，每一类记录为具有依赖关系的记录。In this step, the file identification of each record in the editing log is determined, and the file identification can use a file path prefix, so that each record can be divided into multiple types of records according to the file identification of each recording in the editing log, and each type of record is Records with dependencies.
具体来说，在编辑日志的各记录中包括了具有依赖关系的记录，在采用多线程并发的日志重演log replay的时候，同样需要保证编辑日志edit log里头有依赖关系的各个记录项重演replay的时序关系。假设编辑日志edit log里头有四条记录:(1)创建文件/tmp/file1；(2)为文件/user/xyz/file分配一个数据块；(3)修改文件/tmp/file1的读写权限；(4)设置文件/user/xyz/file的长度和最后修改时间。可知，依赖关系为重演replay记录1之后重演replay记录3，重演replay记录2之后重演replay记录4，从而在日志重演的时候，需要保证记录1重演replay完了之后才能重演replay记录3，同样必须保证记录2重演replay完了之后才能重演replay记录4。采用单线程的时候，重演replay会按照记录1、记录2、记录3、记录4的顺序进行，从而前述的两个时序关系可以得到了保证。但是只是单纯的采用两个多个线程进行日志重演log replay的时候，如果记录1、记录3被分配给两个不同的线程做重演replay，由于没有办法确保两个线程调度和运行的时序，因此也就没有办法确保记录1一定在记录3之前重演replay，同样也没有办法保证记录2一定在记录4之前重演replay。Specifically, each record in the edit log includes records with dependencies. When multi-threaded concurrent log replay is used to replay log replay, it is also necessary to ensure that each record item in the edit log has dependencies to replay. timing relationship. Suppose there are four records in the edit log: (1) create file /tmp/file1; (2) allocate a data block for file /user/xyz/file; (3) modify the read and write permissions of file /tmp/file1; (4) Set the length and last modification time of the file /user/xyz/file. It can be seen that the dependency relationship is replay record 3 after replay record 1, and replay record 4 after replay record 2. Therefore, when replaying the log, it is necessary to ensure that replay record 3 can only be replayed after record 1 is replayed. The record must also be guaranteed 2 The replay record 4 can only be replayed after the replay is finished. When using a single thread, the replay will be performed in the order of record 1, record 2, record 3, and record 4, so that the aforementioned two timing relationships can be guaranteed. However, when simply using two or more threads for log replay, if record 1 and record 3 are assigned to two different threads for replay, there is no way to ensure the timing of scheduling and running of the two threads, so There is no way to ensure that record 1 will replay before record 3, and there is no way to ensure that record 2 will replay before record 4.
从而解决这一个问题，由于具有相同的文件路径前缀的记录具有依赖关系，可以先按各记录的文件路径前缀将编辑日志中的各记录进行分类，将文件路径前缀相同的日志分为同一类记录，那么文件路径前缀不同的记录类别之间就不会有依赖关系，从而不同类别的记录就可以使用不同线程进行重演replay。To solve this problem, since the records with the same file path prefix have dependencies, you can first classify the records in the edit log according to the file path prefix of each record, and classify the logs with the same file path prefix into the same category of records , then there will be no dependency relationship between record categories with different file path prefixes, so records of different categories can use different threads for replay.
举例来说，一个日志log里头有这样一些记录：(1)创建目录/user/a；(2)给文件/tmp/file分配数据块；(3)创建文件/user/a/file；(4)修改文件/tmp/file的文件长度；(5)给文件/user/a/file分配数据块；(6)修改文件/tmp/file的文件主(owner)；(7)修改文件/user/a/file的文件长度。其中，共有两种文件路径前缀“/user/a”和“/tmp/file”，根据这两个文件路径前缀可以将日志log里头的记录分为两类，第一类记录里头包含记录1、3、5、7，第二类记录里头包含记录2、4、6，每一类记录中的各记录具有依赖关系，并且这两类记录之间互相没有依赖关系。For example, there are some records in a log log: (1) create directory /user/a; (2) allocate data blocks to file /tmp/file; (3) create file /user/a/file; (4) ) Modify the file length of the file /tmp/file; (5) allocate data blocks to the file /user/a/file; (6) modify the file owner (owner) of the file /tmp/file; (7) modify the file /user/ The file length of a/file. Among them, there are two file path prefixes "/user/a" and "/tmp/file". According to these two file path prefixes, the records in the log log can be divided into two types. The first type of records includes records 1, 3, 5, 7. The second type of records includes records 2, 4, and 6. Each record in each type of record has a dependency relationship, and there is no dependency relationship between these two types of records.
在步骤S12中，根据划分出的记录的类数，确定用于日志重演的线程数量。In step S12, the number of threads used for log playback is determined according to the number of divided records.
在本步骤中，根据划分出的记录的类数，可以确定出用于日志重演的线程数量。In this step, the number of threads used for log playback can be determined according to the number of divided record categories.
举例来说，举例来说，一个日志log里头有这样一些记录：(1)创建目录/user/a；(2)给文件/tmp/file分配数据块；(3)创建文件/user/a/file；(4)修改文件/tmp/file的文件长度；(5)给文件/user/a/file分配数据块；(6)修改文件/tmp/file的文件主(owner)；(7)修改文件/user/a/file的文件长度。其中，共有两种文件路径前缀“/user/a”和“/tmp/file”，根据这两个文件路径前缀可以将日志log里头的记录分为两类，第一类记录里头包含记录1、3、5、7，第二类记录里头包含记录2、4、6，每一类记录中的各记录具有依赖关系，并且这两类记录之间互相没有依赖关系。从而将各记录分为两类，可以确定用于日志重演的线程数量为2。For example, for example, there are some records in a log log: (1) create directory /user/a; (2) allocate data blocks to file /tmp/file; (3) create file /user/a/ file; (4) modify the file length of the file /tmp/file; (5) allocate data blocks to the file /user/a/file; (6) modify the file owner of the file /tmp/file; (7) modify The file length of the file /user/a/file. Among them, there are two file path prefixes "/user/a" and "/tmp/file". According to these two file path prefixes, the records in the log log can be divided into two types. The first type of records includes records 1, 3, 5, 7. The second type of records includes records 2, 4, and 6. Each record in each type of record has a dependency relationship, and there is no dependency relationship between these two types of records. Therefore, each record is divided into two categories, and the number of threads used for log playback can be determined to be 2.
在步骤S13中，采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。In step S13 , using the determined number of threads, log replay is performed on various records corresponding to each thread one by one.
在本步骤中，可以具有确定出的线程数量的各线程，分别对与各线程一一对应的各类记录进行日志重演。在每个线程里头，日志重演log replay按照记录在编辑log里头出现的顺序进行重演replay，因此这些记录由于依赖关系导致的时序会被保证。In this step, each thread with the determined number of threads may respectively perform log playback on various records corresponding to each thread one by one. In each thread, log replay replays the records in the order they appear in the editing log, so the timing of these records due to dependencies will be guaranteed.
举例来说，将各记录分为两类，可以确定用于日志重演的线程数量为2，可以使用两个不同的线程进行重演replay。采用第一个线程对第一类记录中的记录1、3、5、7依次进行重演replay，同时，采用第二线程对第二类记录里头中的记录2、4、6依次进行重演replay。For example, each record is divided into two categories, the number of threads used for log replay can be determined to be 2, and two different threads can be used for replay. Use the first thread to sequentially replay records 1, 3, 5, and 7 in the first type of records, and use the second thread to sequentially replay records 2, 4, and 6 in the second type of records.
本实施例通过根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；根据划分出的记录的类数，确定用于日志重演的线程数量；采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。从而可以依据编辑日志中的各记录的文件路径前缀，对各记录进行分类，可以保证每一类记录中的各记录的依赖关系；进而可以采用多个线程并发的重演每个记录类别里头的纪录，加快日志重演的速度，可以保证缩短分布式文件系统不可用的时间；并且，编辑日志中的各记录中的时序关系也没有被打破。In this embodiment, according to the file identification of each record in the editing log, each record is divided into at least one type of record, wherein each type of record is a record with a dependency relationship; The number of replaying threads; each thread of the determined thread number is used to perform log replay for various records corresponding to each thread one by one. Therefore, each record can be classified according to the file path prefix of each record in the edit log, and the dependency relationship of each record in each type of record can be guaranteed; then multiple threads can be used to replay the records in each record category concurrently , speeding up the log replay speed can ensure that the unavailable time of the distributed file system is shortened; moreover, the timing relationship among the records in the edit log is not broken.
在上述图1所示的实施例的基础上，图2是根据一示例性实施例示出的一种应用于分布式文件系统的日志重演方法实施例二的流程图，如图2所示，步骤S11，具体包括：On the basis of the embodiment shown in FIG. 1 above, FIG. 2 is a flow chart of Embodiment 2 of a log replay method applied to a distributed file system according to an exemplary embodiment. As shown in FIG. 2 , the steps S11, specifically including:
根据所述编辑日志中的各记录，确定所述编辑日志中的各同步点；将相邻的同步点之间的各记录，确定为同时处理的各记录；根据确定为同时处理的各记录的文件标识，将确定为同时处理的各记录分为至少一类记录。According to each record in the edit log, determine each synchronization point in the edit log; determine each record between adjacent synchronization points as each record processed simultaneously; according to each record determined as simultaneously processed A file identification classifies records determined to be processed concurrently into at least one category of records.
其中，所述同步点为所述编辑日志中的rename操作。Wherein, the synchronization point is the rename operation in the edit log.
在本步骤中，可以首先根据编辑日志中的各记录，确定出编辑日志中的各同步点；其中，同步点可以为编辑日志中的rename操作；然后，将相邻的同步点之间的各记录，确定为同时处理的各记录；进而，就可以根据确定为同时处理的各记录的文件标识，将确定为同时处理的各记录分为至少一类记录。In this step, each synchronization point in the editing log can be determined first according to each record in the editing log; wherein, the synchronization point can be the rename operation in the editing log; then, each synchronization point between adjacent synchronization points The records are the records determined to be processed simultaneously; furthermore, the records determined to be processed simultaneously can be divided into at least one type of records according to the file identifiers of the records determined to be processed simultaneously.
具体来说，在文件系统rename操作中，会牵涉到两个文件路径，一个是源路径，一个是目标路径，从而包含rename的纪录无法单纯的使用实施例一的处理方案。在本实施例中，可以将rename操作作为同步点，即将两个rename操作中间的纪录使用实施例一种的方式进行重演replay，然后重演replay rename操作，然后下一段记录再按照上述方式进行重演replay。举例来说，假定编辑日志edit log里头有以下记录：(1)创建文件/user/xyz/file；(2)创建文件/tmp/file；(3)rename/user/xyz/abc到/tmp/abc；(4)修改/user/xyz/file的访问权限；(5)修改文件/tmp/file的owner；(6)编辑/user/xyz/file的数据；(7)rename/a/b/c到/d/e/f；(8)给文件/user/xyc/file分配新数据块；(9)删除文件/tmp/file。从而，可以确定同步点为记录3、记录7，将记录1、2确定为同时处理的各记录，将记录4、5、6确定为同时处理的各记录，将记录8、9确定为同时处理的各记录，然后根据记录的文件路径前缀，可以将记录4和记录6分为同一类记录。Specifically, in the rename operation of the file system, two file paths are involved, one is the source path, and the other is the target path, so the records containing the rename cannot simply use the processing solution of the first embodiment. In this embodiment, the rename operation can be used as a synchronization point, that is, the record in the middle of the two rename operations is replayed using the method of embodiment 1, and then the replay rename operation is replayed, and then the next record is replayed according to the above method . For example, suppose there are the following records in the edit log: (1) Create file /user/xyz/file; (2) Create file /tmp/file; (3) Rename /user/xyz/abc to /tmp/ abc; (4) Modify the access rights of /user/xyz/file; (5) Modify the owner of the file /tmp/file; (6) Edit the data of /user/xyz/file; (7) rename/a/b/ c to /d/e/f; (8) allocate a new data block to the file /user/xyc/file; (9) delete the file /tmp/file. Therefore, the synchronization point can be determined as record 3 and record 7, records 1 and 2 can be determined as records processed at the same time, records 4, 5 and 6 can be determined as records processed at the same time, and records 8 and 9 can be determined as records processed at the same time , and then according to the file path prefix of the record, record 4 and record 6 can be classified into the same type of record.
步骤S12，具体包括：Step S12 specifically includes:
对各相邻的同步点之间的各类记录，确定各相邻的同步点之间的线程数量。For all types of records between adjacent synchronization points, determine the number of threads between adjacent synchronization points.
在本步骤中，可以针对各相邻的同步点之间的各类记录，确定各相邻的同步点之间的线程数量。In this step, the number of threads between adjacent synchronization points may be determined for various types of records between adjacent synchronization points.
举例来说，假定编辑日志edit log里头有以下记录：(1)创建文件/user/xyz/file；(2)创建文件/tmp/file；(3)rename/user/xyz/abc到/tmp/abc；(4)修改/user/xyz/file的访问权限；(5)修改文件/tmp/file的owner；(6)编辑/user/xyz/file的数据；(7)rename/a/b/c到/d/e/f；(8)给文件/user/xyc/file分配新数据块；(9)删除文件/tmp/file。从而，可以确定同步点为记录3、记录7，将记录1、2确定为同时处理的各记录，将记录4、5、6确定为同时处理的各记录，将记录8、9确定为同时处理的各记录，然后根据记录的文件路径前缀，可以将记录4和记录6分为同一类记录。然后，可以确定针对记录1、2，可以分别采用两个线程分别处理记录1、2；可以确定针对记录3，可以采用一个线程处理记录3；可以确定针对记录4、5、6，可以分别采用一个处理记录4和6，另一个线程的处理记录5；可以确定针对记录7，可以采用一个线程处理记录7；可以确定针对记录8、9，可以分别采用两个线程分别处理记录8、9。从而可以确定出8个线程。For example, suppose there are the following records in the edit log: (1) Create file /user/xyz/file; (2) Create file /tmp/file; (3) Rename /user/xyz/abc to /tmp/ abc; (4) Modify the access rights of /user/xyz/file; (5) Modify the owner of the file /tmp/file; (6) Edit the data of /user/xyz/file; (7) rename/a/b/ c to /d/e/f; (8) allocate a new data block to the file /user/xyc/file; (9) delete the file /tmp/file. Therefore, the synchronization point can be determined as record 3 and record 7, records 1 and 2 can be determined as records processed at the same time, records 4, 5 and 6 can be determined as records processed at the same time, and records 8 and 9 can be determined as records processed at the same time , and then according to the file path prefix of the record, record 4 and record 6 can be classified into the same type of record. Then, it can be determined that for records 1 and 2, two threads can be used to process records 1 and 2 respectively; it can be determined that for record 3, one thread can be used to process record 3; One process records 4 and 6, and another thread processes record 5; it can be determined that for record 7, one thread can be used to process record 7; it can be determined that for records 8 and 9, two threads can be used to process records 8 and 9 respectively. Thus, 8 threads can be determined.
步骤S13，具体包括：Step S13 specifically includes:
重复以下过程，直至对所述编辑日志中的所有记录完成日志重演：采用相邻的同步点之间的线程数量的线程，对于各线程一一对应的各类记录进行日志重演；对同步点进行日志重演。Repeat the following process until all records in the edit log are completed log replay: adopt the threads of the number of threads between adjacent synchronization points to carry out log replay for all kinds of records corresponding to each thread one-to-one; Log repeats.
在本步骤中，针对相邻的同步点之间的线程数量的线程，对于各线程一一对应的各类记录进行日志重演；然后对同步点进行日志重演。重复以上过程，直至对编辑日志中的所有记录完成日志重演。In this step, for the number of threads between adjacent synchronization points, log playback is performed for various records corresponding to each thread; and then log playback is performed for synchronization points. Repeat the above process until log replay is completed for all records in the edit log.
举例来说，假定编辑日志edit log里头有以下记录：(1)创建文件/user/xyz/file；(2)创建文件/tmp/file；(3)rename/user/xyz/abc到/tmp/abc；(4)修改/user/xyz/file的访问权限；(5)修改文件/tmp/file的owner；(6)编辑/user/xyz/file的数据；(7)rename/a/b/c到/d/e/f；(8)给文件/user/xyc/file分配新数据块；(9)删除文件/tmp/file。从而，可以确定同步点为记录3、记录7，将记录1、2确定为同时处理的各记录，将记录4、5、6确定为同时处理的各记录，将记录8、9确定为同时处理的各记录，然后根据记录的文件路径前缀，可以将记录4和记录6分为同一类记录。然后，可以确定针对记录1、2，可以分别采用两个线程分别处理记录1、2；可以确定针对记录3，可以采用一个线程处理记录3；可以确定针对记录4、5、6，可以分别采用一个处理记录4和6，另一个线程的处理记录5；可以确定针对记录7，可以采用一个线程处理记录7；可以确定针对记录8、9，可以分别采用两个线程分别处理记录8、9。从而可以确定出8个线程。然后，重演replay的过程会是这样:第一步，对于记录1、2、3,由于3是一个rename操作,所以它是一个同步点，因此对于3之前的记录会使用两个线程分别重演replay记录1和记录2；第二步重演replay记录3；第三步，对于记录4、5、6、7,由于7是一个rename操作，所以它是一个同步点，并且，记录4和记录6为同一类记录，记录5为另一类记录，因此对于记录7之前的记录会使用两个线程分别重演replay记录4和5，另一个线程重演replay记录6；第四步重演replay记录7；第五步使用两个线程分别重演replay记录8和记录9。整个过程有5个步骤，如果采用单线程的方式共需要9个步骤。For example, suppose there are the following records in the edit log: (1) Create file /user/xyz/file; (2) Create file /tmp/file; (3) Rename /user/xyz/abc to /tmp/ abc; (4) Modify the access rights of /user/xyz/file; (5) Modify the owner of the file /tmp/file; (6) Edit the data of /user/xyz/file; (7) rename/a/b/ c to /d/e/f; (8) allocate a new data block to the file /user/xyc/file; (9) delete the file /tmp/file. Therefore, the synchronization point can be determined as record 3 and record 7, records 1 and 2 can be determined as records processed at the same time, records 4, 5 and 6 can be determined as records processed at the same time, and records 8 and 9 can be determined as records processed at the same time , and then according to the file path prefix of the record, record 4 and record 6 can be classified into the same type of record. Then, it can be determined that for records 1 and 2, two threads can be used to process records 1 and 2 respectively; it can be determined that for record 3, one thread can be used to process record 3; One process records 4 and 6, and another thread processes record 5; it can be determined that for record 7, one thread can be used to process record 7; it can be determined that for records 8 and 9, two threads can be used to process records 8 and 9 respectively. Thus, 8 threads can be determined. Then, the replay process will be as follows: first step, for records 1, 2, and 3, since 3 is a rename operation, it is a synchronization point, so two threads will be used to replay the records before 3 Record 1 and record 2; the second step replays record 3; the third step, for records 4, 5, 6, and 7, since 7 is a rename operation, it is a synchronization point, and record 4 and record 6 are For the same type of records, record 5 is another type of record, so for the records before record 7, two threads will be used to replay the replay records 4 and 5 respectively, and the other thread will replay the replay record 6; the fourth step is to replay the replay record 7; the fifth The step uses two threads to replay record 8 and record 9 respectively. There are 5 steps in the whole process, and a total of 9 steps are required if a single-threaded method is used.
本实施例通过根据编辑日志中的各记录，确定编辑日志中的各同步点；将相邻的同步点之间的各记录，确定为同时处理的各记录；根据确定为同时处理的各记录的文件标识，将确定为同时处理的各记录分为至少一类记录。对各相邻的同步点之间的各类记录，确定各相邻的同步点之间的线程数量。重复以下过程，直至对编辑日志中的所有记录完成日志重演：采用相邻的同步点之间的线程数量的线程，对于各线程一一对应的各类记录进行日志重演；对同步点进行日志重演。从而将相邻的同步点rename操作之间的各记录，确定为同时处理的各记录，然后可以依据编辑日志中的各记录的文件路径前缀，对各记录进行分类，可以保证每一类记录中的各记录的依赖关系；进而可以采用多个线程并发的重演每个记录类别里头的纪录，加快日志重演的速度，可以保证缩短分布式文件系统不可用的时间；并且，编辑日志中的各记录中的时序关系也没有被打破。In this embodiment, each synchronization point in the edit log is determined according to each record in the edit log; each record between adjacent synchronization points is determined as each record processed at the same time; A file identification classifies records determined to be processed concurrently into at least one category of records. For all types of records between adjacent synchronization points, determine the number of threads between adjacent synchronization points. Repeat the following process until the log replay is completed for all the records in the edit log: use the number of threads between adjacent synchronization points to perform log replay for each type of record corresponding to each thread; perform log replay for the synchronization point . Therefore, each record between the adjacent synchronization point rename operations is determined as each record processed at the same time, and then each record can be classified according to the file path prefix of each record in the edit log, which can ensure that each type of record The dependencies of each record; then multiple threads can be used to replay the records in each record category concurrently, speeding up the log replay speed, and ensuring that the time when the distributed file system is unavailable; and, editing each record in the log The timing relationship in is also not broken.
图3是根据一示例性实施例示出的一种应用推荐装置实施例三的框图。参照图3，该装置包括：Fig. 3 is a block diagram of Embodiment 3 of an application recommendation device according to an exemplary embodiment. Referring to Figure 3, the device includes:
分类模块31，被配置为根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；The classification module 31 is configured to classify each record into at least one type of record according to the file identifier of each record in the editing log, wherein each type of record is a record with a dependency relationship;
确定模块32，被配置为根据划分出的记录的类数，确定用于日志重演的线程数量；The determination module 32 is configured to determine the number of threads used for log replay according to the number of divided records;
重演模块33，被配置为采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。The replay module 33 is configured to use the determined number of threads to perform log replay on various records corresponding to each thread one by one.
其中，所述文件标识为文件路径前缀。Wherein, the file identifier is a file path prefix.
所述分类模块31，被具体配置为：The classification module 31 is specifically configured as:
根据所述编辑日志中的各记录，确定所述编辑日志中的各同步点；determining each synchronization point in the edit log based on each record in the edit log;
将相邻的同步点之间的各记录，确定为同时处理的各记录；Determining each record between adjacent synchronization points as each record processed at the same time;
根据确定为同时处理的各记录的文件标识，将确定为同时处理的各记录分为至少一类记录。The records determined to be processed simultaneously are classified into at least one type of records according to the file identifiers of the records determined to be processed simultaneously.
所述确定模块32，被具体配置为：The determination module 32 is specifically configured as:
对各相邻的同步点之间的各类记录，确定各相邻的同步点之间的线程数量。For all types of records between adjacent synchronization points, determine the number of threads between adjacent synchronization points.
所述重演模块33，被具体配置为：The playback module 33 is specifically configured as:
重复以下过程，直至对所述编辑日志中的所有记录完成日志重演：Repeat the following process until log replay is complete for all records in the edit log:
采用相邻的同步点之间的线程数量的线程，对于各线程一一对应的各类记录进行日志重演；Use the number of threads between adjacent synchronization points to perform log replay for all kinds of records corresponding to each thread;
对同步点进行日志重演。Log replay for syncpoints.
其中，所述同步点为所述编辑日志中的rename操作。Wherein, the synchronization point is the rename operation in the edit log.
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。Regarding the apparatus in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments related to the method, and will not be described in detail here.
本实施例通过根据编辑日志中的各记录，确定编辑日志中的各同步点；将相邻的同步点之间的各记录，确定为同时处理的各记录；根据确定为同时处理的各记录的文件标识，将确定为同时处理的各记录分为至少一类记录。对各相邻的同步点之间的各类记录，确定各相邻的同步点之间的线程数量。重复以下过程，直至对编辑日志中的所有记录完成日志重演：采用相邻的同步点之间的线程数量的线程，对于各线程一一对应的各类记录进行日志重演；对同步点进行日志重演。从而将相邻的同步点rename操作之间的各记录，确定为同时处理的各记录，然后可以依据编辑日志中的各记录的文件路径前缀，对各记录进行分类，可以保证每一类记录中的各记录的依赖关系；进而可以采用多个线程并发的重演每个记录类别里头的纪录，加快日志重演的速度，可以保证缩短分布式文件系统不可用的时间；并且，编辑日志中的各记录中的时序关系也没有被打破。In this embodiment, each synchronization point in the edit log is determined according to each record in the edit log; each record between adjacent synchronization points is determined as each record processed at the same time; A file identification classifies records determined to be processed concurrently into at least one category of records. For all types of records between adjacent synchronization points, determine the number of threads between adjacent synchronization points. Repeat the following process until the log replay is completed for all the records in the edit log: use the number of threads between adjacent synchronization points to perform log replay for each type of record corresponding to each thread; perform log replay for the synchronization point . Therefore, each record between the adjacent synchronization point rename operations is determined as each record processed at the same time, and then each record can be classified according to the file path prefix of each record in the edit log, which can ensure that each type of record The dependencies of each record; then multiple threads can be used to replay the records in each record category concurrently, speeding up the log replay speed, and ensuring that the time when the distributed file system is unavailable; and, editing each record in the log The timing relationship in is also not broken.
图4是根据一示例性实施例示出的一种终端的实体的框图。参照图4，该终端可以具体实现为：处理器71，以及被配置为存储处理器可执行指令的存储器72；Fig. 4 is a block diagram showing entities of a terminal according to an exemplary embodiment. Referring to FIG. 4, the terminal may be specifically implemented as: a processor 71, and a memory 72 configured to store instructions executable by the processor;
其中，所述处理器71被配置为：根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；根据划分出的记录的类数，确定用于日志重演的线程数量；采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。Wherein, the processor 71 is configured to: divide each record into at least one type of record according to the file identifier of each record in the editing log, wherein each type of record is a record with a dependency relationship; The number of classes determines the number of threads used for log playback; each thread of the determined number of threads is used to perform log playback for various records corresponding to each thread one by one.
在上述实施例中，应理解，该处理器可以是中央处理单元(英文：CentralProcessing Unit，简称：CPU)，还可以是其他通用处理器、数字信号处理器(英文：DigitalSignal Processor，简称：DSP)、专用集成电路(英文：Application Specific IntegratedCircuit，简称：ASIC)等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等，而前述的存储器可以是只读存储器(英文：read-only memory，缩写：ROM)、随机存取存储器(英文：random access memory，简称：RAM)、快闪存储器、硬盘或者固态硬盘。SIM卡也称为用户身份识别卡、智能卡，数字移动电话机必须装上此卡方能使用。即在电脑芯片上存储了数字移动电话客户的信息，加密的密钥以及用户的电话簿等内容。结合本发明实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。In the foregoing embodiments, it should be understood that the processor may be a central processing unit (English: Central Processing Unit, referred to as: CPU), and may also be other general-purpose processors, digital signal processors (English: DigitalSignal Processor, referred to as: DSP) , Application Specific Integrated Circuit (English: Application Specific Integrated Circuit, referred to as: ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., and the aforementioned memory can be a read-only memory (English: read-only memory, abbreviated: ROM), random access memory (English: : random access memory, referred to as: RAM), flash memory, hard disk or solid state disk. SIM card is also known as subscriber identification card, smart card, digital mobile phone must be equipped with this card before it can be used. That is, the information of the digital mobile phone customer, the encrypted key and the user's phone book are stored on the computer chip. The steps of the methods disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.
关于上述实施例中的终端，其中各个模块执行操作的具体方式已经在有关该方法和装置的实施例中进行了详细描述，此处将不做详细阐述说明。With regard to the terminal in the foregoing embodiments, the specific manner in which each module executes operations has been described in detail in the embodiments of the method and apparatus, and will not be described in detail here.
本实施例通过根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；根据划分出的记录的类数，确定用于日志重演的线程数量；采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。从而可以依据编辑日志中的各记录的文件路径前缀，对各记录进行分类，可以保证每一类记录中的各记录的依赖关系；进而可以采用多个线程并发的重演每个记录类别里头的纪录，加快日志重演的速度，可以保证缩短分布式文件系统不可用的时间；并且，编辑日志中的各记录中的时序关系也没有被打破。In this embodiment, according to the file identification of each record in the editing log, each record is divided into at least one type of record, wherein each type of record is a record with a dependency relationship; The number of replaying threads; each thread of the determined thread number is used to perform log replay for various records corresponding to each thread one by one. Therefore, each record can be classified according to the file path prefix of each record in the edit log, and the dependency relationship of each record in each type of record can be guaranteed; then multiple threads can be used to replay the records in each record category concurrently , speeding up the log replay speed can ensure that the unavailable time of the distributed file system is shortened; moreover, the timing relationship among the records in the edit log is not broken.
图5是根据一示例性实施例示出的一种终端设备800的框图。例如，终端设备800可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等。Fig. 5 is a block diagram showing a terminal device 800 according to an exemplary embodiment. For example, the terminal device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.
参照图5，终端设备800可以包括以下一个或多个组件：处理组件802，存储器804，电源组件806，多媒体组件808，音频组件810，输入/输出(I/O)的接口812，传感器组件814，以及通信组件816。5, terminal device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814 , and the communication component 816.
处理组件802通常控制终端设备800的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件802可以包括一个或多个处理器820来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件802可以包括一个或多个模块，便于处理组件802和其他组件之间的交互。例如，处理组件802可以包括多媒体模块，以方便多媒体组件808和处理组件802之间的交互。The processing component 802 generally controls the overall operations of the terminal device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802 .
存储器804被配置为存储各种类型的数据以支持在终端设备800的操作。这些数据的示例包括用于在终端设备800上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器804可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。The memory 804 is configured to store various types of data to support operations at the terminal device 800 . Examples of such data include instructions for any application or method operating on the terminal device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
电源组件806为终端设备800的各种组件提供电力。电源组件806可以包括电源管理系统，一个或多个电源，及其他与为终端设备800生成、管理和分配电力相关联的组件。The power supply component 806 provides power to various components of the terminal device 800 . Power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for end device 800 .
多媒体组件808包括在所述终端设备800和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件808包括一个前置摄像头和/或后置摄像头。当终端设备800处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。The multimedia component 808 includes a screen providing an output interface between the terminal device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the terminal device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.
音频组件810被配置为输出和/或输入音频信号。例如，音频组件810包括一个麦克风(MIC)，当终端设备800处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器804或经由通信组件816发送。在一些实施例中，音频组件810还包括一个扬声器，用于输出音频信号。The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive an external audio signal when the terminal device 800 is in an operation mode, such as a call mode, a recording mode and a voice recognition mode. Received audio signals may be further stored in memory 804 or sent via communication component 816 . In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.
I/O接口812为处理组件802和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.
传感器组件814包括一个或多个传感器，用于为终端设备800提供各个方面的状态评估。例如，传感器组件814可以检测到终端设备800的打开/关闭状态，组件的相对定位，例如所述组件为终端设备800的显示器和小键盘，传感器组件814还可以检测终端设备800或终端设备800一个组件的位置改变，用户与终端设备800接触的存在或不存在，终端设备800方位或加速/减速和终端设备800的温度变化。传感器组件814可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件814还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件814还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。The sensor component 814 includes one or more sensors for providing status assessment of various aspects of the terminal device 800 . For example, the sensor component 814 can detect the opening/closing state of the terminal device 800, the relative positioning of the components, for example, the components are the display and the keypad of the terminal device 800, and the sensor component 814 can also detect the terminal device 800 or a terminal device 800 Changes in the positions of components, presence or absence of user contact with the terminal device 800 , orientation or acceleration/deceleration of the terminal device 800 and temperature changes of the terminal device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
通信组件816被配置为便于终端设备800和其他设备之间有线或无线方式的通信。终端设备800可以接入基于通信标准的无线网络，如WiFi，2G或3G，或它们的组合。在一个示例性实施例中，通信组件816经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件816还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。The communication component 816 is configured to facilitate wired or wireless communications between the terminal device 800 and other devices. The terminal device 800 can access a wireless network based on communication standards, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.
在示例性实施例中，终端设备800可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。In an exemplary embodiment, terminal device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable A programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the methods described above.
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例如包括指令的存储器804，上述指令可由终端设备800的处理器820执行以完成上述方法。例如，所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium including instructions, such as a memory 804 including instructions, which can be executed by the processor 820 of the terminal device 800 to complete the above method. For example, the non-transitory computer readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
一种非临时性计算机可读存储介质，当所述存储介质中的指令由移动终端的处理器执行时，使得移动终端能够执行一种应用于分布式文件系统的日志重演方法，所述方法包括：A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal can execute a log playback method applied to a distributed file system, the method comprising :
根据编辑日志中各记录的文件标识，将各记录分为至少一类记录，其中，每一类记录为具有依赖关系的记录；According to the file identification of each record in the editing log, each record is divided into at least one type of record, wherein each type of record is a record with a dependency relationship;
根据划分出的记录的类数，确定用于日志重演的线程数量；Determine the number of threads used for log replay according to the number of recorded records;
采用确定出的线程数量的各线程，对与各线程一一对应的各类记录进行日志重演。Each thread of the determined number of threads is used to perform log replay for various records corresponding to each thread one by one.
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本发明的其它实施方案。本申请旨在涵盖本发明的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本发明的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本发明的真正范围和精神由下面的权利要求指出。Other embodiments of the invention will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present invention, these modifications, uses or adaptations follow the general principles of the present invention and include common knowledge or conventional technical means in the technical field not disclosed in this disclosure . The specification and examples are to be considered exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
应当理解的是，本发明并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本发明的范围仅由所附的权利要求来限制。It should be understood that the present invention is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710109052.6ACN106919679B (en) | 2017-02-27 | 2017-02-27 | Log replay method, device and terminal applied to distributed file system |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710109052.6ACN106919679B (en) | 2017-02-27 | 2017-02-27 | Log replay method, device and terminal applied to distributed file system |
| Publication Number | Publication Date |
|---|---|
| CN106919679Atrue CN106919679A (en) | 2017-07-04 |
| CN106919679B CN106919679B (en) | 2019-12-13 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710109052.6AActiveCN106919679B (en) | 2017-02-27 | 2017-02-27 | Log replay method, device and terminal applied to distributed file system |
| Country | Link |
|---|---|
| CN (1) | CN106919679B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109885543A (en)* | 2018-12-24 | 2019-06-14 | 航天信息股份有限公司 | Log processing method and device based on big data cluster |
| CN110795318A (en)* | 2018-08-01 | 2020-02-14 | 阿里巴巴集团控股有限公司 | Data processing method and device and electronic equipment |
| WO2022002103A1 (en)* | 2020-06-30 | 2022-01-06 | 华为技术有限公司 | Method for playing back log on data node, data node, and system |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103064761A (en)* | 2012-12-24 | 2013-04-24 | 华为技术有限公司 | Data synchronization method, device and system |
| CN105045917A (en)* | 2015-08-20 | 2015-11-11 | 北京百度网讯科技有限公司 | Example-based distributed data recovery method and device |
| CN106294626A (en)* | 2016-08-02 | 2017-01-04 | 极道科技（北京）有限公司 | A kind of method that parallel playback file system redoes log |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103064761A (en)* | 2012-12-24 | 2013-04-24 | 华为技术有限公司 | Data synchronization method, device and system |
| CN105045917A (en)* | 2015-08-20 | 2015-11-11 | 北京百度网讯科技有限公司 | Example-based distributed data recovery method and device |
| CN106294626A (en)* | 2016-08-02 | 2017-01-04 | 极道科技（北京）有限公司 | A kind of method that parallel playback file system redoes log |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110795318A (en)* | 2018-08-01 | 2020-02-14 | 阿里巴巴集团控股有限公司 | Data processing method and device and electronic equipment |
| CN110795318B (en)* | 2018-08-01 | 2023-05-02 | 阿里云计算有限公司 | Data processing method and device and electronic equipment |
| CN109885543A (en)* | 2018-12-24 | 2019-06-14 | 航天信息股份有限公司 | Log processing method and device based on big data cluster |
| WO2022002103A1 (en)* | 2020-06-30 | 2022-01-06 | 华为技术有限公司 | Method for playing back log on data node, data node, and system |
| Publication number | Publication date |
|---|---|
| CN106919679B (en) | 2019-12-13 |
| Publication | Publication Date | Title |
|---|---|---|
| CN107370772B (en) | Account login method, device and computer-readable storage medium | |
| US20170150290A1 (en) | Method and device for information push | |
| CN104484200A (en) | Method and device for updating firmware | |
| JP2016526244A (en) | Application program processing method, apparatus, program, and storage medium | |
| CN105845124A (en) | Audio processing method and device | |
| CN105426386A (en) | File synchronization method and apparatus, and terminal device | |
| CN112632184A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN104636164A (en) | Generating method and device for starting page | |
| CN105183571A (en) | Function calling method and device | |
| CN106919679B (en) | Log replay method, device and terminal applied to distributed file system | |
| CN108491282A (en) | The method and apparatus that webpage and operating system are communicated | |
| CN107463372B (en) | Data-driven page updating method and device | |
| CN115062012B (en) | Data processing method, device, storage medium and electronic equipment | |
| CN114003558A (en) | Data archiving method, device, equipment and storage medium | |
| CN106060253B (en) | Information presentation method and device | |
| CN106354657B (en) | Register access method, device and system | |
| CN104765646A (en) | Assembly transferring method and device | |
| CN111290882B (en) | Data file backup method, data file backup device and electronic equipment | |
| CN116489247A (en) | Device and method for editing random network protocol message programmable in operation | |
| CN108021399B (en) | Memo processing method, device, equipment and storage medium | |
| CN111104110A (en) | Global style sharing method, global style sharing device and electronic device | |
| CN114116075B (en) | A method, device, electronic device and storage medium for obtaining page parameters | |
| CN109710334A (en) | Dynamic loading method, device and storage medium | |
| CN112650594B (en) | Resource processing method and device | |
| CN112306254B (en) | Expression processing method, device and medium |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |