CN111984588A - Object storage life cycle management method, device and system - Google Patents
Object storage life cycle management method, device and systemDownload PDFInfo
- Publication number
- CN111984588A CN111984588ACN202010657074.8ACN202010657074ACN111984588ACN 111984588 ACN111984588 ACN 111984588ACN 202010657074 ACN202010657074 ACN 202010657074ACN 111984588 ACN111984588 ACN 111984588A
- Authority
- CN
- China
- Prior art keywords
- storage space
- life cycle
- configuration information
- filtering
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/113—Details of archiving
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本发明涉及对象存储领域，具体涉及一种对象存储生命周期管理方法、装置及系统。The present invention relates to the field of object storage, in particular to a method, device and system for the management of an object storage life cycle.
背景技术Background technique
对象存储会存储海量数据，为降低存储成本，大部分对象存储系统都会设置生命周期管理功能，用于对满足生命周期管理条件的Object进行归档存储(Archive，低频访问存储)或对满足生命管路周期条件的Object直接进行删除。Object storage will store massive amounts of data. In order to reduce storage costs, most object storage systems will set up a lifecycle management function, which is used to archive and store objects that meet the conditions of The object of the periodic condition is deleted directly.
目前对象存储系统中的文件生命周期管理方法主要有两种，一种是面向单个Object的生命周期，在上传时由用户指定过期时间，到期后将不可再访问。另一种是设置整个Bucket(存储空间)的生命周期管理配置，按照配置来过滤与管理该存储空间下的所有Object，对满足配置条件的Object由标准存储(Standard)转化为归档存储或直接进行删除。At present, there are two main methods of file life cycle management in the object storage system. One is the life cycle for a single object. The user specifies the expiration time when uploading, and it will no longer be accessible after the expiration. The other is to set the life cycle management configuration of the entire bucket (storage space), filter and manage all objects under the storage space according to the configuration, and convert the objects that meet the configuration conditions from standard storage (Standard) to archive storage or directly. delete.
在实现本发明过程中，申请人发现现有技术中至少存在如下问题：In the process of realizing the present invention, the applicant found that there are at least the following problems in the prior art:
在目前的对象存储中，在面对海量文件时对单个Object设置生命周期较为繁琐，因此大多数是针对整个Bucket进行生命周期管理配置。但是各个Bucket下Object数量不一致，可归档或删除的Object数量也不相同，当需要在一定时间内完成归档或删除任务时，在服务器、网络、数据库资源等有限的情况下，就需要对不同的Bucket用不同的速度来处理。但是目前的生命周期管理无法直接进行清理速度的动态调整。In the current object storage, it is cumbersome to set the life cycle of a single object in the face of massive files, so most of them are configured for the life cycle management of the entire bucket. However, the number of objects in each bucket is inconsistent, and the number of objects that can be archived or deleted is also different. When the task of archiving or deleting needs to be completed within a certain period of time, in the case of limited server, network, database resources, etc., it is necessary to Buckets are processed at different speeds. However, the current life cycle management cannot directly adjust the cleaning speed dynamically.
发明内容SUMMARY OF THE INVENTION
本发明实施例提供一种对象存储生命周期管理方法、装置及系统，控制模块不断监听为Bucket设置的生命配置周期，可通过修改该Bucket的生命周期配置实时控制各个Bucket的归档删除速度。The embodiments of the present invention provide an object storage life cycle management method, device, and system. The control module continuously monitors the life configuration cycle set for the bucket, and can control the archive deletion speed of each bucket in real time by modifying the life cycle configuration of the bucket.
为达上述目的，一方面，本发明实施例提供一种对象存储生命周期的管理方法，所述方法包括：In order to achieve the above purpose, on the one hand, an embodiment of the present invention provides a method for managing the object storage life cycle, the method comprising:
服务器从配置中心处拉取所有存储空间对应的生命周期配置信息；所述配置中心存储每一个存储空间对应的生命周期配置信息，所述生命周期配置信息包括过滤条件和执行参数；The server pulls the life cycle configuration information corresponding to all storage spaces from the configuration center; the configuration center stores the life cycle configuration information corresponding to each storage space, and the life cycle configuration information includes filter conditions and execution parameters;
服务器在处理任一存储空间下的对象时，获取该存储空间对应的分布式锁，成功获取分布式锁之后，实时监听配置中心中该存储空间对应的生命周期配置信息的变化情况；When processing objects in any storage space, the server obtains the distributed lock corresponding to the storage space, and after successfully obtaining the distributed lock, monitors the changes of the life cycle configuration information corresponding to the storage space in the configuration center in real time;
基于最新的该存储空间对应的生命周期配置信息，按照过滤条件对该存储空间下的对象进行过滤，并对满足过滤条件的对象按照执行参数进行相应处理。Based on the latest life cycle configuration information corresponding to the storage space, the objects under the storage space are filtered according to the filter conditions, and the objects that meet the filter conditions are processed correspondingly according to the execution parameters.
另一方面，本发明实施例还提供一种对象存储生命周期管理装置，设置在服务器中，包括：On the other hand, the embodiment of the present invention also provides an object storage life cycle management device, which is set in the server and includes:
处理模块，用于从配置中心处拉取所有存储空间对应的生命周期配置信息；所述配置中心存储每一个存储空间对应的生命周期配置信息，所述生命周期配置信息包括过滤条件和执行参数；以及，在处理任一存储空间下的对象时，获取该存储空间对应的分布式锁，成功获取分布式锁之后，启动本模块包括的控制模块、过滤模块和归档删除模块；a processing module, configured to pull the life cycle configuration information corresponding to all storage spaces from the configuration center; the configuration center stores the life cycle configuration information corresponding to each storage space, and the life cycle configuration information includes filter conditions and execution parameters; And, when processing objects under any storage space, obtain the distributed lock corresponding to the storage space, and after successfully obtaining the distributed lock, start the control module, filter module and archive deletion module included in this module;
控制模块，用于实时监听配置中心中该存储空间对应的生命周期配置信息的变化情况；The control module is used to monitor the changes of the life cycle configuration information corresponding to the storage space in the configuration center in real time;
过滤模块，用于基于最新的该存储空间对应的生命周期配置信息，按照过滤条件对该存储空间下的对象进行过滤，The filtering module is used to filter the objects under the storage space according to the filter conditions based on the latest life cycle configuration information corresponding to the storage space,
归档删除模块，用于对满足过滤条件的对象按照执行参数进行相应处理。The archive deletion module is used to perform corresponding processing on the objects that meet the filtering conditions according to the execution parameters.
另一方面，本发明实施例还提供一种对象存储生命周期管理系统，包括配置中心和多个服务器，每个服务器中设置有对象存储生命周期管理装置，其中：On the other hand, an embodiment of the present invention also provides an object storage life cycle management system, including a configuration center and a plurality of servers, each server is provided with an object storage life cycle management device, wherein:
所述对象存储生命周期管理装置，用于从配置中心处拉取所有存储空间对应的生命周期配置信息；在处理任一存储空间下的对象时，获取该存储空间对应的分布式锁，成功获取分布式锁之后，实时监听配置中心中该存储空间对应的生命周期配置信息的变化情况；基于最新的该存储空间对应的生命周期配置信息，按照过滤条件对该存储空间下的对象进行过滤，对满足过滤条件的对象按照执行参数进行相应处理；The object storage life cycle management device is used to pull the life cycle configuration information corresponding to all storage spaces from the configuration center; when processing objects in any storage space, obtain the distributed lock corresponding to the storage space, and successfully obtain the After the distributed lock, monitor the changes of the life cycle configuration information corresponding to the storage space in the configuration center in real time; based on the latest life cycle configuration information corresponding to the storage space, filter the objects under the storage space according to the Objects that meet the filter conditions are processed according to the execution parameters;
所述配置中心，用于存储每一个存储空间对应的生命周期配置信息，所述生命周期配置信息包括过滤条件和执行参数。The configuration center is used to store life cycle configuration information corresponding to each storage space, and the life cycle configuration information includes filter conditions and execution parameters.
上述技术方案具有如下有益效果：控制模块不断监听为Bucket设置的生命配置周期，可通过修改该Bucket的生命周期配置实时控制各个Bucket的归档删除速度。在处理Bucket下的Object时，首先获取对应Bucket的分布式锁(Distributed locks)，获取失败则直接结束，目的在高可用，同时保证对同一bucket各操作(过滤、删除、处理)的一致性，使得在同一时间内只用有一个组控制模块对同一bucket进行操作。避免同时有两个进程在处理同一个bucket下的文件。The above technical solution has the following beneficial effects: the control module continuously monitors the life configuration cycle set for the bucket, and can control the archive deletion speed of each bucket in real time by modifying the life cycle configuration of the bucket. When processing objects under a bucket, the distributed locks corresponding to the bucket are first acquired, and if the acquisition fails, the process ends immediately. The purpose is high availability, and at the same time, the consistency of each operation (filtering, deleting, and processing) on the same bucket is ensured. So that only one group control module operates on the same bucket at the same time. Avoid having two processes working on files in the same bucket at the same time.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.
图1是本发明实施例的对象存储生命周期的管理方法的流程图；1 is a flowchart of a method for managing an object storage life cycle according to an embodiment of the present invention;
图2是本发明实施例的对象存储生命周期的管理装置的结构图；2 is a structural diagram of an apparatus for managing the object storage life cycle according to an embodiment of the present invention;
图3是本发明实施例的对象存储生命周期的管理系统的结构图；3 is a structural diagram of an object storage life cycle management system according to an embodiment of the present invention;
图4是本发明实施例的对象存储生命周期的实施逻辑图。FIG. 4 is an implementation logic diagram of an object storage life cycle according to an embodiment of the present invention.
具体实施方式Detailed ways
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
如图1所示结合本发明的实施例，提供一种对象存储生命周期的管理方法，所述方法包括：As shown in FIG. 1 in conjunction with an embodiment of the present invention, a method for managing the object storage life cycle is provided, and the method includes:
S101：服务器从配置中心处拉取所有存储空间对应的生命周期配置信息；所述配置中心存储每一个存储空间对应的生命周期配置信息，所述生命周期配置信息包括过滤条件和执行参数；S101: The server pulls the life cycle configuration information corresponding to all storage spaces from the configuration center; the configuration center stores the life cycle configuration information corresponding to each storage space, and the life cycle configuration information includes filter conditions and execution parameters;
S102：服务器在处理任一存储空间下的对象时，获取该存储空间对应的分布式锁，成功获取分布式锁之后，实时监听配置中心中该存储空间对应的生命周期配置信息的变化情况；S102: When processing objects in any storage space, the server acquires a distributed lock corresponding to the storage space, and after successfully acquiring the distributed lock, monitors in real time changes in the life cycle configuration information corresponding to the storage space in the configuration center;
S103：基于最新的该存储空间对应的生命周期配置信息，按照过滤条件对该存储空间下的对象进行过滤，并对满足过滤条件的对象按照执行参数进行相应处理。S103: Based on the latest life cycle configuration information corresponding to the storage space, filter objects under the storage space according to the filter conditions, and perform corresponding processing on the objects that satisfy the filter conditions according to the execution parameters.
优选地，所述生命周期配置信息还包括：该生命周期配置信息发生变化后的生效时间；Preferably, the life cycle configuration information further includes: an effective time after the life cycle configuration information is changed;
所述过滤条件包括如下可选字段：前缀名prefix、后缀名suffix、文件类型filetype、过期时间expires、存活时间lifetime；The filtering conditions include the following optional fields: prefix name prefix, suffix name suffix, file type filetype, expiration time expires, and survival time lifetime;
所述执行参数包括如下可选字段：执行动作action、执行速度speed、停止/开始执行动作stop/start，所述执行动作包括归档存储或删除。The execution parameter includes the following optional fields: execution action action, execution speed speed, stop/start execution action stop/start, and the execution action includes archive storage or deletion.
优选地，所述执行速度表示执行动作所需的第一线程数量；Preferably, the execution speed represents the number of first threads required to execute the action;
步骤103具体包括：Step 103 specifically includes:
根据最新的该存储空间对应的生命周期配置信息内的执行速度将执行动作所需的线程数量调整为所述第一线程数量。The number of threads required to execute the action is adjusted to the first number of threads according to the execution speed in the latest life cycle configuration information corresponding to the storage space.
优选地，所述执行参数的可选字段还包括过滤速度，过滤速度表示进行过滤时所需的第二线程数量；Preferably, the optional field of the execution parameter further includes a filtering speed, and the filtering speed represents the number of second threads required for filtering;
步骤103具体包括：Step 103 specifically includes:
通过携带过滤条件的线程对该存储空间下的对象进行过滤；以及，根据最新的该存储空间对应的生命周期配置信息内的过滤速度将携带过滤条件的线程数量调整为所述第二线程数量。Filter the objects under the storage space by the thread carrying the filter condition; and adjust the number of threads carrying the filter condition to the number of the second thread according to the filtering speed in the latest life cycle configuration information corresponding to the storage space.
优选地，还包括：Preferably, it also includes:
S104：在服务器获取该存储空间对应的分布式锁之后，采用该分布式锁锁定相应的存储空间；S104: After the server acquires the distributed lock corresponding to the storage space, use the distributed lock to lock the corresponding storage space;
S105：当服务器完成对任一存储空间下的对象的处理之后，释放锁定该存储空间的分布式锁；或者，当服务器不能正常运行时释放锁定该存储空间的分布式锁。S105: After the server completes the processing of the objects in any storage space, the distributed lock for locking the storage space is released; or, when the server cannot operate normally, the distributed lock for locking the storage space is released.
如图2所示结合本发明的实施例，提供一种对象存储生命周期管理装置，设置在服务器中，包括：As shown in FIG. 2 in conjunction with an embodiment of the present invention, an object storage life cycle management device is provided, which is set in a server and includes:
处理模块21，用于从配置中心处拉取所有存储空间对应的生命周期配置信息；所述配置中心存储每一个存储空间对应的生命周期配置信息，所述生命周期配置信息包括过滤条件和执行参数；以及，在处理任一存储空间下的对象时，获取该存储空间对应的分布式锁，成功获取分布式锁之后，启动控制模块22、过滤模块23和归档删除模块24；The processing module 21 is used to pull the life cycle configuration information corresponding to all storage spaces from the configuration center; the configuration center stores the life cycle configuration information corresponding to each storage space, and the life cycle configuration information includes filter conditions and execution parameters And, when processing the object under any storage space, obtain the corresponding distributed lock of this storage space, after successfully obtaining the distributed lock, start control module 22, filter module 23 and archive deletion module 24;
控制模块22，用于实时监听配置中心中该存储空间对应的生命周期配置信息的变化情况；The control module 22 is used to monitor the changes of the life cycle configuration information corresponding to the storage space in the configuration center in real time;
过滤模块23，用于基于最新的该存储空间对应的生命周期配置信息，按照过滤条件对该存储空间下的对象进行过滤，The filtering module 23 is used to filter the objects under the storage space according to the filtering conditions based on the latest life cycle configuration information corresponding to the storage space,
归档删除模块24，用于对满足过滤条件的对象按照执行参数进行相应处理。The archive deletion module 24 is configured to perform corresponding processing on the objects satisfying the filtering conditions according to the execution parameters.
优选地，所述生命周期配置信息还包括：该生命周期配置信息发生变化后的生效时间；Preferably, the life cycle configuration information further includes: an effective time after the life cycle configuration information is changed;
所述过滤条件包括如下可选字段：前缀名prefix、后缀名suffix、文件类型filetype、过期时间expires、存活时间lifetime；The filtering conditions include the following optional fields: prefix name prefix, suffix name suffix, file type filetype, expiration time expires, and survival time lifetime;
所述执行参数包括如下可选字段：执行动作action、执行速度speed、停止/开始执行动作stop/start，所述执行动作包括归档存储或删除。The execution parameter includes the following optional fields: execution action action, execution speed speed, stop/start execution action stop/start, and the execution action includes archive storage or deletion.
优选地，所述执行速度表示执行动作所需的第一线程数量；Preferably, the execution speed represents the number of first threads required to execute the action;
所述归档删除模块24，具体用于：The archive deletion module 24 is specifically used for:
根据最新的该存储空间对应的生命周期配置信息内的执行速度，通过控制模块22将执行动作所需的线程数量调整为所述第一线程数量。According to the execution speed in the latest life cycle configuration information corresponding to the storage space, the control module 22 adjusts the number of threads required to perform the action to be the first number of threads.
优选地，所述执行参数的可选字段还包括过滤速度，过滤速度表示进行过滤时所需的第二线程数量；Preferably, the optional field of the execution parameter further includes a filtering speed, and the filtering speed represents the number of second threads required for filtering;
所述过滤模块23，具体用于：The filtering module 23 is specifically used for:
通过携带过滤条件的线程对该存储空间下的对象进行过滤；以及，根据最新的该存储空间对应的生命周期配置信息内的过滤速度，通过控制模块22将携带过滤条件的线程数量调整为所述第二线程数量。Filter the objects under the storage space by the thread carrying the filter condition; and, according to the latest filtering speed in the life cycle configuration information corresponding to the storage space, the control module 22 adjusts the number of threads carrying the filter condition to the described Number of second threads.
优选地，所述处理模块21还用于：获取该存储空间对应的分布式锁之后，采用该分布式锁锁定相应的存储空间；Preferably, the processing module 21 is further configured to: after acquiring the distributed lock corresponding to the storage space, use the distributed lock to lock the corresponding storage space;
当完成对任一存储空间下的对象的处理之后，释放锁定该存储空间的分布式锁；或者，当服务器不能正常运行时释放锁定该存储空间的分布式锁。After completing the processing of objects in any storage space, the distributed lock for locking the storage space is released; or, when the server fails to run normally, the distributed lock for locking the storage space is released.
本发明的实施例还提供一种对象存储生命周期管理系统，包括配置中心和多个服务器，每个服务器中设置有对象存储生命周期管理装置，其中：The embodiment of the present invention also provides an object storage life cycle management system, including a configuration center and a plurality of servers, each server is provided with an object storage life cycle management device, wherein:
所述对象存储生命周期管理装置，用于从配置中心处拉取所有存储空间对应的生命周期配置信息；在处理任一存储空间下的对象时，获取该存储空间对应的分布式锁，成功获取分布式锁之后，实时监听配置中心中该存储空间对应的生命周期配置信息的变化情况；基于最新的该存储空间对应的生命周期配置信息，按照过滤条件对该存储空间下的对象进行过滤，对满足过滤条件的对象按照执行参数进行相应处理；The object storage life cycle management device is used to pull the life cycle configuration information corresponding to all storage spaces from the configuration center; when processing objects under any storage space, obtain the distributed lock corresponding to the storage space, and successfully obtain the After the distributed lock, monitor the changes of the life cycle configuration information corresponding to the storage space in the configuration center in real time; based on the latest life cycle configuration information corresponding to the storage space, filter the objects under the storage space according to the Objects that meet the filter conditions are processed according to the execution parameters;
所述配置中心，用于存储每一个存储空间对应的生命周期配置信息，所述生命周期配置信息包括过滤条件和执行参数。The configuration center is used to store life cycle configuration information corresponding to each storage space, and the life cycle configuration information includes filter conditions and execution parameters.
本发明所取得的有益效果如下：The beneficial effects obtained by the present invention are as follows:
由控制模块不断监听为Bucket设置的生命配置周期，可通过修改该Bucket的生命周期配置实时控制各个Bucket的归档删除速度。The control module continuously monitors the life configuration cycle set for the bucket, and can control the archive deletion speed of each bucket in real time by modifying the life cycle configuration of the bucket.
当生命周期配置改变时，通过增加归档删除模块的数量可实时改变对对象的处理动作，所以可以实时实现停止归档或删除，从而可减少因生命周期配置错误，在修改生命周期配置后也无法阻止归档或删除对象而造成的丢失大量Object。从而使得Bucket生命周期配置实时生效，实现了生命周期管理可实时调整处理速度，以及实时停止。When the life cycle configuration is changed, the processing action of the object can be changed in real time by increasing the number of archive deletion modules, so it is possible to stop the archive or delete in real time, thereby reducing the life cycle configuration errors, which cannot be prevented after modifying the life cycle configuration. A large number of objects are lost due to archiving or deleting objects. As a result, the bucket life cycle configuration takes effect in real time, and the life cycle management can adjust the processing speed in real time and stop in real time.
增加了Bucket下后缀名匹配、文件类型等匹配规则，可更灵活的面对各种业务需求；避免了现有技术中目前的生命周期管理系统支持的生命周期配置信息少、且不支持后缀和文件类型等匹配规则的缺点。Added matching rules such as suffix name matching and file type under Bucket, which can more flexibly meet various business needs; avoid the current life cycle management system in the existing technology, which supports less life cycle configuration information and does not support suffix and Disadvantages of matching rules such as file type.
在处理Bucket下的Object时，首先获取对应Bucket的分布式锁(Distributedlocks)，获取失败则直接结束，目的在高可用，同时保证对同一bucket各操作(过滤、删除、处理)的一致性，使得在同一时间内只用有一个组控制模块对同一bucket进行操作。避免同时有两个进程在处理同一个bucket下的文件。When processing objects under a bucket, first obtain the distributed locks (Distributedlocks) corresponding to the bucket. If the acquisition fails, it will end immediately. The purpose is high availability. Only one group control module operates on the same bucket at the same time. Avoid having two processes working on files in the same bucket at the same time.
根据具体生命周期配置的情况实时调整各模块(过滤模块、删除归档模块)的数量，各尽其责，能够实现成本优化。下面结合具体的应用实例对本发明实施例上述技术方案进行详细说明，实施过程中没有介绍到的技术细节，可以参考前文的相关描述。According to the specific life cycle configuration, the number of each module (filtering module, deleting and archiving module) is adjusted in real time, and each performs its own responsibilities, which can achieve cost optimization. The foregoing technical solutions of the embodiments of the present invention will be described in detail below with reference to specific application examples. For technical details not introduced in the implementation process, reference may be made to the foregoing related descriptions.
本发明为一种对象存储生命周期管理方法和装置，涉及对象存储技术领域，尤其涉及一种大规模分布式对象存储系统中的文件(Object)生命周期(Lifecycle)管理方法。The invention relates to an object storage life cycle management method and device, and relates to the technical field of object storage, in particular to a file (Object) life cycle (Lifecycle) management method in a large-scale distributed object storage system.
本发明为用于对象存储系统的一种生命周期管理方法，可以在大量Bucket及其海量Object中，按照指定的生命周期配置自动检索Bucket、归档或删除指定配置的Object，可以控制各个Bucket的处理速度，同时可以实时停止。如附图3所示，配置中心里存储对所有Bucket的生命周期配置，生命周期配置如表1所示，当Bucket下的Object满足表中定义的条件时，则执行配置的动作(归档或者删除)。The present invention is a life cycle management method for an object storage system, which can automatically retrieve Buckets, archive or delete objects with a specified configuration in a large number of Buckets and their massive objects according to the specified life cycle configuration, and can control the processing of each Bucket speed, and can be stopped in real time. As shown in Figure 3, the configuration center stores the life cycle configuration of all buckets. The life cycle configuration is shown in Table 1. When the Object under the Bucket meets the conditions defined in the table, the configured action (archive or delete) is executed. ).
表1Table 1
各台服务器到配置中心分别拉取所有配置信息，即为一个Bucket在某段时间内对应设置一种生命周期配置。然后由处理模块按照配置信息对该Bucket下的所有Object进行过滤与归档删除；过滤与归档删除的具体操作为：由生命周期配置的action字段所决定的，这个字段体现归档或删除。action字段由对象存储的用户自行设定；即满足生命周期配置内设定条件的对象文件，就要执行action所代表的动作。Each server pulls all configuration information from the configuration center, that is, sets a corresponding life cycle configuration for a Bucket within a certain period of time. Then, the processing module filters and archives and deletes all objects under the bucket according to the configuration information; the specific operations of filtering and archive deletion are: determined by the action field of the life cycle configuration, which reflects the archive or deletion. The action field is set by the user of the object storage; that is, the object file that satisfies the conditions set in the life cycle configuration must execute the action represented by the action.
处理模块结构如附图4所示，在处理Bucket下的Object时，首先获取对应Bucket的分布式锁(Distributed locks)，获取失败则直接结束。当成功获取锁之后，启动三个模块：The structure of the processing module is shown in Figure 4. When processing objects under a bucket, the distributed locks (Distributed locks) corresponding to the bucket are first obtained, and if the acquisition fails, the process ends directly. When the lock is successfully acquired, three modules are started:
控制模块负责不断监听(即检索)配置中心对该Bucket的生命周期配置，当发生变化时，实时控制过滤模块与归档删除模块，以达到控制速度或实时停止，具体为：过滤模块负责按照该Bucket的生命周期配置来对此Bucket下的所有Object进行过滤，将满足条件的Object发送至归档删除模块进行处理；其中归档处理可理解为：某些文件用不到，但是确不想删除想一直保存，但用户为了降低存储成本，将这些文件换到其他介质进行存储，比如从现存的硬盘换到光盘内。The control module is responsible for constantly monitoring (that is, retrieving) the configuration center's life cycle configuration of the Bucket. When changes occur, it controls the filtering module and the archive deletion module in real time to achieve the control speed or real-time stop. Specifically: the filtering module is responsible for following the Bucket The life cycle configuration of the bucket is used to filter all objects under this bucket, and the objects that meet the conditions are sent to the archive deletion module for processing; the archive processing can be understood as: some files are not used, but you really don't want to delete them and want to keep them all the time. However, in order to reduce storage costs, users change these files to other media for storage, such as changing from an existing hard disk to an optical disc.
归档删除模块负责把Object转化为归档存储或者直接删除。The archive deletion module is responsible for converting Objects into archive storage or direct deletion.
举例为：就像一个文件目录，我要找出这个目录下所有文件中，文件名以test为开头的文件；For example: just like a file directory, I want to find out the files whose file names start with test among all the files in this directory;
bucket就是可理解为目录，一个object可理解为目录下的一个文件，先需要过滤出bucket内的文件名以test为开头的文件；A bucket can be understood as a directory, and an object can be understood as a file in a directory. First, it is necessary to filter out the files whose file names start with test in the bucket;
过滤条件为：1.文件名以test开头的文件2.创建时间在2020.1.2之前3.到当前时间为止已经存在了超过1个月4.文件名以.mp4为结尾的文件The filter conditions are: 1. The file name starts with test 2. The creation time is before 2020.1.2 3. The current time has existed for more than 1 month 4. The file name ends with .mp4
执行参数为：5.这些文件执行删除；The execution parameters are: 5. These files are deleted;
当bucket下的某些对象满足过滤条件内定义的所有规则时，就可以将这些对象过滤出来，根据过滤出来的对象名称和bucket名称，就可对object执行参数里所定义的动作：删除。When some objects under the bucket meet all the rules defined in the filter conditions, these objects can be filtered out. According to the filtered object name and bucket name, the action defined in the parameters can be executed on the object: delete.
还有，控制模块为主进程，过滤模块和归档删除模块为控制模块的子进程(即线程)，主进程可以子进程的运行、可以控制子进程的数量。在生命周期配置发生变化时，控制模块的主进程调整过滤模块线程的数量、归档删除模块线程数量使得线程数量与用于过滤的对象数量、与归档删除对象的数量相匹配，以便于能够控制过滤速度、归档和删除的速度，所以也能够将根据之前生命周期配置所进行的过滤、归档和删除及时停止。Also, the control module is the main process, the filtering module and the archive deletion module are sub-processes (ie threads) of the control module, and the main process can run the sub-processes and control the number of sub-processes. When the life cycle configuration changes, the main process of the control module adjusts the number of threads in the filtering module and the number of threads in the archive deletion module so that the number of threads matches the number of objects used for filtering and the number of objects in archive deletion, so as to be able to control the filtering speed, archiving, and deleting, so filtering, archiving, and deleting based on previous lifecycle configurations can also be stopped in time.
通常过滤模块一个就够了，因为查数据库很快，1亿的文件也就5分钟过滤完成了；但是归档和删除的速度较慢，所以以通过控制模块控制后面归档或删除的数量来控制速度。Usually one filter module is enough, because the database is very fast, and 100 million files are filtered in 5 minutes; but the speed of archiving and deletion is slow, so the speed is controlled by controlling the number of subsequent archives or deletions through the control module. .
还有，控制模块里(主进程)设置一个线程到配置中心分别拉取所有配置信息。Also, set a thread in the control module (main process) to the configuration center to pull all configuration information respectively.
实时监听配置中心中该存储空间对应的生命周期配置信息的变化情况；比如1s为间隔的实时监听生命周期配置信息的变化情况，并且某一bucket的生命周期配置信息发生变化时，通常为该生命周期配置信息设置生效时间，通常在修改时间之后的预设时间间隔之后生效，从而保证在监听到生命配置变化后，主进程有足够的时间相应的增加、或减少子线程的数量，或者直接停止主进程，所有都停止；总的来说就是，能够保证高可用，配置可调，速度可调，开关可调。Monitor the changes in the life cycle configuration information corresponding to the storage space in the configuration center in real time; for example, monitor the changes in the life cycle configuration information in real time at intervals of 1s, and when the life cycle configuration information of a bucket changes, it is usually the life cycle configuration information of the bucket. The period configuration information sets the effective time, which usually takes effect after the preset time interval after the modification time, so as to ensure that after monitoring the life configuration change, the main process has enough time to increase or decrease the number of child threads accordingly, or stop directly. For the main process, everything is stopped; in general, it can ensure high availability, adjustable configuration, adjustable speed, and adjustable switches.
本发明所取得的有益效果如下：The beneficial effects obtained by the present invention are as follows:
由控制模块不断监听为Bucket设置的生命配置周期，可通过修改该Bucket的生命周期配置实时控制各个Bucket的归档删除速度。The control module continuously monitors the life configuration cycle set for the bucket, and can control the archive deletion speed of each bucket in real time by modifying the life cycle configuration of the bucket.
当生命周期配置改变时，通过增加归档删除模块的数量可实时改变对对象的处理动作，所以可以实时实现停止归档或删除，从而可减少因生命周期配置错误，在修改生命周期配置后也无法阻止归档或删除对象而造成的丢失大量Object。从而使得Bucket生命周期配置实时生效，实现了生命周期管理可实时调整处理速度，以及实时停止。When the life cycle configuration is changed, the processing action of the object can be changed in real time by increasing the number of archive deletion modules, so it is possible to stop the archive or delete in real time, thereby reducing the life cycle configuration errors, which cannot be prevented after modifying the life cycle configuration. A large number of objects are lost due to archiving or deleting objects. As a result, the bucket life cycle configuration takes effect in real time, and the life cycle management can adjust the processing speed in real time and stop in real time.
增加了Bucket下后缀名匹配、文件类型等匹配规则，可更灵活的面对各种业务需求；避免了现有技术中目前的生命周期管理系统支持的生命周期配置信息少、且不支持后缀和文件类型等匹配规则的缺点。Added matching rules such as suffix name matching and file type under Bucket, which can more flexibly meet various business needs; avoid the current life cycle management system in the existing technology, which supports less life cycle configuration information and does not support suffix and Disadvantages of matching rules such as file type.
在处理Bucket下的Object时，首先获取对应Bucket的分布式锁(Distributedlocks)，获取失败则直接结束，目的在高可用，同时保证对同一bucket各操作(过滤、删除、处理)的一致性，使得在同一时间内只用有一个组控制模块对同一bucket进行操作。避免同时有两个进程在处理同一个bucket下的文件。When processing objects under a bucket, first obtain the distributed locks (Distributedlocks) corresponding to the bucket. If the acquisition fails, it will end immediately. The purpose is high availability. Only one group control module operates on the same bucket at the same time. Avoid having two processes working on files in the same bucket at the same time.
根据具体生命周期配置的情况实时调整各模块(过滤模块、删除归档模块)的数量，各尽其责，能够实现成本优化。According to the specific life cycle configuration, the number of each module (filtering module, deleting and archiving module) is adjusted in real time, and each performs its own responsibilities, which can achieve cost optimization.
应该明白，公开的过程中的步骤的特定顺序或层次是示例性方法的实例。基于设计偏好，应该理解，过程中的步骤的特定顺序或层次可以在不脱离本公开的保护范围的情况下得到重新安排。所附的方法权利要求以示例性的顺序给出了各种步骤的要素，并且不是要限于所述的特定顺序或层次。It is understood that the specific order or hierarchy of steps in the disclosed processes is an example of a sample approach. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.
在上述的详细描述中，各种特征一起组合在单个的实施方案中，以简化本公开。不应该将这种公开方法解释为反映了这样的意图，即，所要求保护的主题的实施方案需要比清楚地在每个权利要求中所陈述的特征更多的特征。相反，如所附的权利要求书所反映的那样，本发明处于比所公开的单个实施方案的全部特征少的状态。因此，所附的权利要求书特此清楚地被并入详细描述中，其中每项权利要求独自作为本发明单独的优选实施方案。In the foregoing Detailed Description, various features are grouped together in a single embodiment for the purpose of simplifying the disclosure. This method of disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are expressly recited in each claim. Rather, as the following claims reflect, present invention lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby expressly incorporated into the Detailed Description, with each claim standing on its own as a separate preferred embodiment of this invention.
为使本领域内的任何技术人员能够实现或者使用本发明，上面对所公开实施例进行了描述。对于本领域技术人员来说；这些实施例的各种修改方式都是显而易见的，并且本文定义的一般原理也可以在不脱离本公开的精神和保护范围的基础上适用于其它实施例。因此，本公开并不限于本文给出的实施例，而是与本申请公开的原理和新颖性特征的最广范围相一致。The disclosed embodiments are described above to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit and scope of this disclosure. Thus, the present disclosure is not intended to be limited to the embodiments set forth herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
上文的描述包括一个或多个实施例的举例。当然，为了描述上述实施例而描述部件或方法的所有可能的结合是不可能的，但是本领域普通技术人员应该认识到，各个实施例可以做进一步的组合和排列。因此，本文中描述的实施例旨在涵盖落入所附权利要求书的保护范围内的所有这样的改变、修改和变型。此外，就说明书或权利要求书中使用的术语“包含”，该词的涵盖方式类似于术语“包括”，就如同“包括，”在权利要求中用作衔接词所解释的那样。此外，使用在权利要求书的说明书中的任何一个术语“或者”是要表示“非排它性的或者”。The above description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods in order to describe the above embodiments, but one of ordinary skill in the art will recognize that further combinations and permutations of the various embodiments are possible. Accordingly, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of the appended claims. Furthermore, with respect to the term "comprising," as used in the specification or claims, the word is encompassed in a manner similar to the term "comprising," as if "comprising," were interpreted as a conjunction in the claims. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or."
本领域技术人员还可以了解到本发明实施例列出的各种说明性逻辑块(illustrative logical block)，单元，和步骤可以通过电子硬件、电脑软件，或两者的结合进行实现。为清楚展示硬件和软件的可替换性(interchangeability)，上述的各种说明性部件(illustrative components)，单元和步骤已经通用地描述了它们的功能。这样的功能是通过硬件还是软件来实现取决于特定的应用和整个系统的设计要求。本领域技术人员可以对于每种特定的应用，可以使用各种方法实现所述的功能，但这种实现不应被理解为超出本发明实施例保护的范围。Those skilled in the art may also understand that various illustrative logical blocks (illustrative logical blocks), units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. To clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units and steps described above have generally described their functions. Whether such functionality is implemented in hardware or software depends on the specific application and overall system design requirements. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be construed as exceeding the protection scope of the embodiments of the present invention.
本发明实施例中所描述的各种说明性的逻辑块，或单元都可以通过通用处理器，数字信号处理器，专用集成电路(ASIC)，现场可编程门阵列或其它可编程逻辑装置，离散门或晶体管逻辑，离散硬件部件，或上述任何组合的设计来实现或操作所描述的功能。通用处理器可以为微处理器，可选地，该通用处理器也可以为任何传统的处理器、控制器、微控制器或状态机。处理器也可以通过计算装置的组合来实现，例如数字信号处理器和微处理器，多个微处理器，一个或多个微处理器联合一个数字信号处理器核，或任何其它类似的配置来实现。The various illustrative logic blocks, or units described in the embodiments of the present invention can be implemented by general-purpose processors, digital signal processors, application specific integrated circuits (ASICs), field programmable gate arrays or other programmable logic devices, discrete Gate or transistor logic, discrete hardware components, or any combination of the above are designed to implement or operate the functions described. A general-purpose processor may be a microprocessor, or alternatively, the general-purpose processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors in combination with a digital signal processor core, or any other similar configuration. accomplish.
本发明实施例中所描述的方法或算法的步骤可以直接嵌入硬件、处理器执行的软件模块、或者这两者的结合。软件模块可以存储于RAM存储器、闪存、ROM存储器、EPROM存储器、EEPROM存储器、寄存器、硬盘、可移动磁盘、CD-ROM或本领域中其它任意形式的存储媒介中。示例性地，存储媒介可以与处理器连接，以使得处理器可以从存储媒介中读取信息，并可以向存储媒介存写信息。可选地，存储媒介还可以集成到处理器中。处理器和存储媒介可以设置于ASIC中，ASIC可以设置于用户终端中。可选地，处理器和存储媒介也可以设置于用户终端中的不同的部件中。The steps of the method or algorithm described in the embodiments of the present invention may be directly embedded in hardware, a software module executed by a processor, or a combination of the two. Software modules may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. Illustratively, a storage medium may be coupled to the processor such that the processor may read information from, and store information in, the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium may be provided in the ASIC, and the ASIC may be provided in the user terminal. Alternatively, the processor and the storage medium may also be provided in different components in the user terminal.
在一个或多个示例性的设计中，本发明实施例所描述的上述功能可以在硬件、软件、固件或这三者的任意组合来实现。如果在软件中实现，这些功能可以存储与电脑可读的媒介上，或以一个或多个指令或代码形式传输于电脑可读的媒介上。电脑可读媒介包括电脑存储媒介和便于使得让电脑程序从一个地方转移到其它地方的通信媒介。存储媒介可以是任何通用或特殊电脑可以接入访问的可用媒体。例如，这样的电脑可读媒体可以包括但不限于RAM、ROM、EEPROM、CD-ROM或其它光盘存储、磁盘存储或其它磁性存储装置，或其它任何可以用于承载或存储以指令或数据结构和其它可被通用或特殊电脑、或通用或特殊处理器读取形式的程序代码的媒介。此外，任何连接都可以被适当地定义为电脑可读媒介，例如，如果软件是从一个网站站点、服务器或其它远程资源通过一个同轴电缆、光纤电缆、双绞线、数字用户线(DSL)或以例如红外、无线和微波等无线方式传输的也被包含在所定义的电脑可读媒介中。所述的碟片(disk)和磁盘(disc)包括压缩磁盘、镭射盘、光盘、DVD、软盘和蓝光光盘，磁盘通常以磁性复制数据，而碟片通常以激光进行光学复制数据。上述的组合也可以包含在电脑可读媒介中。In one or more exemplary designs, the above functions described in the embodiments of the present invention may be implemented in hardware, software, firmware, or any combination of the three. If implemented in software, the functions may be stored on, or transmitted over, a computer-readable medium in the form of one or more instructions or code. Computer-readable media includes computer storage media and communication media that facilitate the transfer of a computer program from one place to another. Storage media can be any available media that a general-purpose or special-purpose computer can access. For example, such computer-readable media may include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other device that can be used to carry or store instructions or data structures and Other media in the form of program code that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection is properly defined as a computer-readable medium, for example, if software is transmitted from a website site, server or other remote source over a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) Or transmitted by wireless means such as infrared, wireless, and microwave are also included in the definition of computer-readable media. The disks and disks include compact disks, laser disks, optical disks, DVDs, floppy disks and blu-ray disks. Disks usually reproduce data magnetically, while discs generally reproduce data optically with lasers. Combinations of the above can also be included in computer readable media.
以上所述的具体实施方式，对本发明的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本发明的具体实施方式而已，并不用于限定本发明的保护范围，凡在本发明的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The specific embodiments described above further describe the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims (11)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010657074.8ACN111984588A (en) | 2020-07-09 | 2020-07-09 | Object storage life cycle management method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010657074.8ACN111984588A (en) | 2020-07-09 | 2020-07-09 | Object storage life cycle management method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111984588Atrue CN111984588A (en) | 2020-11-24 |
Family
ID=73438600
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010657074.8APendingCN111984588A (en) | 2020-07-09 | 2020-07-09 | Object storage life cycle management method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111984588A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113076290A (en)* | 2021-04-12 | 2021-07-06 | 百果园技术(新加坡)有限公司 | File deletion method, device, equipment, system and storage medium |
| CN113157414A (en)* | 2021-04-19 | 2021-07-23 | 星辰天合（北京）数据科技有限公司 | Task processing method and device, nonvolatile storage medium and processor |
| CN113392127A (en)* | 2021-01-20 | 2021-09-14 | 腾讯科技（深圳）有限公司 | Data management method and device |
| CN115826880A (en)* | 2023-02-14 | 2023-03-21 | 北京大道云行科技有限公司 | Local caching method, device, system, medium and equipment of object storage system |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100145917A1 (en)* | 2002-07-30 | 2010-06-10 | Stored Iq, Inc. | System, method and apparatus for enterprise policy management |
| US20160086260A1 (en)* | 2014-09-19 | 2016-03-24 | Amazon Technologies, Inc. | Lifecycle transitions in log-coordinated data stores |
| KR101761128B1 (en)* | 2016-04-18 | 2017-07-25 | 전자부품연구원 | Fluorescence optical system for biosensor |
| US20180372640A1 (en)* | 2016-03-31 | 2018-12-27 | Fujifilm Corporation | Imaging device and imaging method |
| CN109542874A (en)* | 2018-11-15 | 2019-03-29 | 用友网络科技股份有限公司 | Deriving method, guiding device, server, terminal and readable storage medium storing program for executing |
| US20190219810A1 (en)* | 2016-09-30 | 2019-07-18 | Olympus Corporation | Observation apparatus |
| CN110175159A (en)* | 2019-05-29 | 2019-08-27 | 京东数字科技控股有限公司 | Method of data synchronization and system for object storage cluster |
| CN111124643A (en)* | 2019-12-20 | 2020-05-08 | 浪潮电子信息产业股份有限公司 | Task deletion scheduling method, system and related device in distributed storage |
| CN111339033A (en)* | 2020-02-29 | 2020-06-26 | 苏州浪潮智能科技有限公司 | Method, system and terminal for realizing object storage life cycle |
| US20210325309A1 (en)* | 2020-04-21 | 2021-10-21 | Essen Instruments, Inc. D/B/A Essen Bioscience, Inc. | Optical Module with Three or More Color Fluorescent Light Sources and Methods for Use Thereof |
| CN114414546A (en)* | 2022-01-28 | 2022-04-29 | 福州大学 | A high-throughput liquid-phase biomolecule detection method and device |
- 2020
- 2020-07-09CNCN202010657074.8Apatent/CN111984588A/enactivePending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100145917A1 (en)* | 2002-07-30 | 2010-06-10 | Stored Iq, Inc. | System, method and apparatus for enterprise policy management |
| US20160086260A1 (en)* | 2014-09-19 | 2016-03-24 | Amazon Technologies, Inc. | Lifecycle transitions in log-coordinated data stores |
| US20180372640A1 (en)* | 2016-03-31 | 2018-12-27 | Fujifilm Corporation | Imaging device and imaging method |
| KR101761128B1 (en)* | 2016-04-18 | 2017-07-25 | 전자부품연구원 | Fluorescence optical system for biosensor |
| US20190219810A1 (en)* | 2016-09-30 | 2019-07-18 | Olympus Corporation | Observation apparatus |
| CN109542874A (en)* | 2018-11-15 | 2019-03-29 | 用友网络科技股份有限公司 | Deriving method, guiding device, server, terminal and readable storage medium storing program for executing |
| CN110175159A (en)* | 2019-05-29 | 2019-08-27 | 京东数字科技控股有限公司 | Method of data synchronization and system for object storage cluster |
| CN111124643A (en)* | 2019-12-20 | 2020-05-08 | 浪潮电子信息产业股份有限公司 | Task deletion scheduling method, system and related device in distributed storage |
| CN111339033A (en)* | 2020-02-29 | 2020-06-26 | 苏州浪潮智能科技有限公司 | Method, system and terminal for realizing object storage life cycle |
| US20210325309A1 (en)* | 2020-04-21 | 2021-10-21 | Essen Instruments, Inc. D/B/A Essen Bioscience, Inc. | Optical Module with Three or More Color Fluorescent Light Sources and Methods for Use Thereof |
| CN114414546A (en)* | 2022-01-28 | 2022-04-29 | 福州大学 | A high-throughput liquid-phase biomolecule detection method and device |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113392127A (en)* | 2021-01-20 | 2021-09-14 | 腾讯科技（深圳）有限公司 | Data management method and device |
| CN113076290A (en)* | 2021-04-12 | 2021-07-06 | 百果园技术(新加坡)有限公司 | File deletion method, device, equipment, system and storage medium |
| CN113076290B (en)* | 2021-04-12 | 2024-01-30 | 百果园技术(新加坡)有限公司 | File deletion method, device, equipment, system and storage medium |
| CN113157414A (en)* | 2021-04-19 | 2021-07-23 | 星辰天合（北京）数据科技有限公司 | Task processing method and device, nonvolatile storage medium and processor |
| CN113157414B (en)* | 2021-04-19 | 2023-11-21 | 北京星辰天合科技股份有限公司 | Task processing method and device, nonvolatile storage medium and processor |
| CN115826880A (en)* | 2023-02-14 | 2023-03-21 | 北京大道云行科技有限公司 | Local caching method, device, system, medium and equipment of object storage system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111984588A (en) | Object storage life cycle management method, device and system | |
| CN112789606B (en) | Data redistribution method, device and system | |
| US11275763B2 (en) | Storage constrained synchronization of shared content items | |
| CN109144785B (en) | Method and apparatus for backing up data | |
| US20170308599A1 (en) | Storage Constrained Synchronization Engine | |
| US20080270699A1 (en) | System for controlling spinning of disk | |
| CN110750497B (en) | Data scheduling system | |
| CN103605585A (en) | Intelligent backup method based on data discovery | |
| CN107918650A (en) | A kind of snapshot creation method, device, system and computer-readable recording medium | |
| US8943364B2 (en) | Appliance for storing, managing and analyzing problem determination artifacts | |
| WO2024067832A1 (en) | Data processing method and apparatus, electronic device, and storage medium | |
| WO2020224236A1 (en) | Apparatus, method and system for blockchain data processing, and storage medium | |
| US20170308600A1 (en) | Storage Constrained Synchronization Engine | |
| WO2025123848A1 (en) | Data import method and apparatus, electronic device, storage medium, and program product | |
| CN110019131B (en) | Method and device for multi-disk service | |
| US10713226B1 (en) | Managing data using archiving | |
| CN111708835A (en) | Blockchain data storage method and device | |
| CN114258532A (en) | Apparatus and method for consolidating backup policies | |
| CN116467336A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN116149903A (en) | Distributed file automatic backup method based on multiple network protocols | |
| CN111966533B (en) | Electronic file management method, device, computer equipment and storage medium | |
| CN114756588A (en) | Redis cache management method, apparatus, electronic device and storage medium | |
| CN111338894A (en) | Remote monitoring method, system, equipment and medium | |
| CN114372044A (en) | Data processing method and device | |
| CN113836112A (en) | Data migration method, system, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right | Effective date of registration:20230407 Address after:Room 501-502, 5/F, Sina Headquarters Scientific Research Building, Block N-1 and N-2, Zhongguancun Software Park, Dongbei Wangxi Road, Haidian District, Beijing, 100193 Applicant after:Sina Technology (China) Co.,Ltd. Address before:100193 7th floor, scientific research building, Sina headquarters, plot n-1, n-2, Zhongguancun Software Park, Dongbei Wangxi Road, Haidian District, Beijing, 100193 Applicant before:Sina.com Technology (China) Co.,Ltd. | |
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication | Application publication date:20201124 |