CN114610679A - Storage device, data storage method thereof and cloud storage system - Google Patents
Storage device, data storage method thereof and cloud storage systemDownload PDFInfo
- Publication number
- CN114610679A CN114610679ACN202110298190.XACN202110298190ACN114610679ACN 114610679 ACN114610679 ACN 114610679ACN 202110298190 ACN202110298190 ACN 202110298190ACN 114610679 ACN114610679 ACN 114610679A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- file
- cloud
- cloud storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/162—Delete operations
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1737—Details of further file system functions for reducing power consumption or coping with limited storage space, e.g. in mobile devices
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Translated fromChinese
Description
Translated fromChinese技术领域technical field
本申请涉及存储技术领域，具体涉及一种存储设备及其数据存储方法、一种云端存储系统。The present application relates to the field of storage technologies, and in particular to a storage device, a data storage method thereof, and a cloud storage system.
背景技术Background technique
随着科技的发展，信息系统正在从以计算为核心转向以数据为核心，海量存储将成为国家信息化的核心组成部分。目前，信息资源呈几何级数的速度增长，涉及的不仅包括文本、图像，还有语音、视频等多媒体信息。因此，众多部门和单位都开始部署自己的数据中心，实现海量数据的集中式存储、分析和共享。现有的数据中心中的数据存储方式是云存储技术，非常适合海量信息的存储，因此能为信息中心提供很方便的解决方案，实现数据的集中式存储和共享。With the development of science and technology, the information system is changing from computing as the core to data as the core, and mass storage will become the core component of national informatization. At present, information resources are growing at a geometric rate, involving not only text and images, but also multimedia information such as voice and video. Therefore, many departments and units have begun to deploy their own data centers to achieve centralized storage, analysis and sharing of massive data. The data storage method in the existing data center is cloud storage technology, which is very suitable for the storage of massive information, so it can provide a very convenient solution for the information center to realize the centralized storage and sharing of data.
但是现有的此类的数据存储方式太过粗糙，其并没有对数据在本地化的存储和云存储进行有效管理，其不但耗费了大量的设备投入，而且对数据缺乏有效地管理，基于此，本发明提出一种存储设备及其数据存储方法、云端存储系统来解决数据的存储解决数据在存储设备中的合理化存储，提高存储设备的利用率。However, the existing data storage methods of this type are too rough, and they do not effectively manage data in localized storage and cloud storage, which not only consumes a lot of equipment investment, but also lacks effective management of data. Based on this The present invention proposes a storage device, a data storage method thereof, and a cloud storage system to solve the storage of data, solve the rational storage of data in the storage device, and improve the utilization rate of the storage device.
发明内容SUMMARY OF THE INVENTION
鉴于此，本申请提供一种存储设备及其数据存储方法以及一种云端存储系统，以解决现有的存储设备存储空间有限的问题。In view of this, the present application provides a storage device, a data storage method thereof, and a cloud storage system to solve the problem of limited storage space of the existing storage device.
本申请提供一种存储设备，包括：存储模块，用于存储本地的数据文件以及网络地址索引文件，所述网络地址索引文件指向云端存储空间内对应的云端存储地址；数据传输模块，用于在设备本地与云端存储系统之间进行数据传输，包括：将本地的满足预设条件的数据文件上传至云端存储系统，并形成与编辑所述云端存储地址的网络地址索引文件。The application provides a storage device, comprising: a storage module for storing local data files and a network address index file, wherein the network address index file points to a corresponding cloud storage address in the cloud storage space; a data transmission module for storing in the cloud storage space The data transmission between the local device and the cloud storage system includes: uploading the local data files that meet the preset conditions to the cloud storage system, and forming and editing the network address index file of the cloud storage address.
可选的，所述数据传输模块还用于响应对网络地址索引文件的操作，访问对应的云端存储地址和/或下载对应的云端存储地址内的数据文件。Optionally, the data transmission module is further configured to access the corresponding cloud storage address and/or download the data file in the corresponding cloud storage address in response to the operation on the network address index file.
可选的，还包括：数据分析模块，连接至所述存储模块和数据传输模块，用于监测所述存储设备的存储空间大小，当监测到剩余存储空间小于预设值时，遵循预设条件对存储空间内的数据进行清理。Optionally, it also includes: a data analysis module, connected to the storage module and the data transmission module, for monitoring the storage space size of the storage device, and following preset conditions when monitoring that the remaining storage space is less than a preset value Clean up the data in the storage space.
可选的，所述数据分析模块还用于在对存储空间内的数据进行清理的同时在日志系统记录数据清理相关的操作信息。Optionally, the data analysis module is further configured to record operation information related to data cleaning in the log system while cleaning the data in the storage space.
可选的，所述数据分析模块还用于将存储设备中的数据文件通过字符串变换算法，生成每一份数据文件的唯一的MD5值，并且将待上传的数据文件的MD5值与云端存储系统内的各个数据文件的MD5值进行比较，当云端存储系统内不存在相同的MD5值时，控制所述数据传输模块将所述待上传得数据文件上传至云端存储系统。Optionally, the data analysis module is also used to generate the unique MD5 value of each data file by passing the data file in the storage device through a string transformation algorithm, and to compare the MD5 value of the data file to be uploaded with the cloud storage. The MD5 values of each data file in the system are compared, and when the same MD5 value does not exist in the cloud storage system, the data transmission module is controlled to upload the to-be-uploaded data file to the cloud storage system.
可选的，所述数据传输模块用于通过外部已联网的终端设备与所述云端存储系统之间建立数据传输路径。Optionally, the data transmission module is configured to establish a data transmission path between an external networked terminal device and the cloud storage system.
可选的，所述存储模块包括缓存区，用于在所述数据传输模块进行数据传输时，对数据进行缓存。Optionally, the storage module includes a buffer area for buffering data when the data transmission module performs data transmission.
可选的，所述预设条件包括：文件存储时间超过第一阈值、文件大小超过第二阈值、距离最近一次被打开的时间超过第三阈值以及设备存储容量小于第四阈值中的至少一个。Optionally, the preset conditions include at least one of: file storage time exceeds a first threshold, file size exceeds a second threshold, time since last opening exceeds a third threshold, and device storage capacity is less than a fourth threshold.
可选的，所述存储设备上传的数据文件被集中或分布式存储于所述云端存储系统内；所述数据传输模块在下载云端存储地址内的数据文件时，用于直接从云端存储系统下载完整的文件，或者获取多个数据块后对所述多个数据块进行数据拼接获取完整的数据文件。Optionally, the data files uploaded by the storage device are stored in the cloud storage system in a centralized or distributed manner; when the data transmission module downloads the data files in the cloud storage address, it is used to directly download the data files from the cloud storage system. A complete file, or after obtaining multiple data blocks, data splicing is performed on the multiple data blocks to obtain a complete data file.
本申请还提供一种存储设备的数据存储方法，包括：判断设备本地存储的数据文件是否满足预设条件；将满足预设条件的数据文件上传至云端存储系统，并获取该数据文件在云端存储空间内的云端存储地址；在设备本地存储与上传的数据文件对应的网络地址索引文件，所述网络地址索引文件指向该数据文件的云端存储地址。The present application also provides a data storage method for a storage device, including: judging whether a data file stored locally by the device meets a preset condition; uploading the data file that meets the preset condition to a cloud storage system, and obtaining the data file and storing it in the cloud The cloud storage address in the space; the network address index file corresponding to the uploaded data file is stored locally on the device, and the network address index file points to the cloud storage address of the data file.
可选的，还包括：响应对网络地址索引文件的操作，访问对应的云端存储地址和/或下载对应的云端存储地址内的数据文件。Optionally, the method further includes: in response to the operation on the network address index file, accessing the corresponding cloud storage address and/or downloading the data file in the corresponding cloud storage address.
可选的，还包括：将存储设备中的数据文件通过字符串变换算法，生成每一份数据文件的唯一的MD5值，并且将待上传的数据文件的MD5值与云端存储系统内的各个数据文件的MD5值进行比较，当云端存储系统内不存在相同的MD5值时，将所述待上传得数据文件上传至云端存储系统。Optionally, it also includes: generating a unique MD5 value of each data file by passing the data files in the storage device through a string transformation algorithm, and comparing the MD5 value of the data file to be uploaded with each data in the cloud storage system. The MD5 values of the files are compared, and when the same MD5 value does not exist in the cloud storage system, the data file to be uploaded is uploaded to the cloud storage system.
可选的，还包括：监测所述存储设备的存储空间大小，当监测到剩余存储空间小于预设值时，遵循预设条件对存储空间内的数据进行清理。Optionally, the method further includes: monitoring the size of the storage space of the storage device, and when it is detected that the remaining storage space is less than a preset value, clearing the data in the storage space according to a preset condition.
可选的，所述数据分析模块还用于在对存储空间内的数据进行清理的同时在日志系统记录数据清理相关的操作信息。Optionally, the data analysis module is further configured to record operation information related to data cleaning in the log system while cleaning the data in the storage space.
本申请还提供一种存储设备，包括：处理器；存储器，存储有能够被所述处理器执行的计算机程序，所述计算机程序被执行时能够执行如上述任一项权利要求所述的数据存储方法。The present application further provides a storage device, comprising: a processor; a memory, which stores a computer program that can be executed by the processor, and when the computer program is executed, can execute the data storage according to any one of the preceding claims method.
本申请还提供一种云端存储系统，包括：云端存储空间，用于存储如上述任一项所述的存储设备上传的数据文件；云端控制模块，与所述云端存储空间连接，用于与存储设备之间建立数据传输连接，为所述存储设备上传的数据文件分配对应的云端存储地址，以及将所述云端存储地址反馈至所述存储设备端。The present application also provides a cloud storage system, including: a cloud storage space for storing data files uploaded by the storage device described in any of the above; a cloud control module, connected to the cloud storage space, for connecting with the storage A data transmission connection is established between the devices, a corresponding cloud storage address is allocated to the data file uploaded by the storage device, and the cloud storage address is fed back to the storage device.
本申请上述存储设备通过将数据文件上传至云端存储系统，并通过索引文件进行访问，解决现有存储设备的存储空间有限的问题。The above-mentioned storage device of the present application solves the problem of limited storage space of the existing storage device by uploading the data file to the cloud storage system and accessing it through the index file.
附图说明Description of drawings
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.
图1是本申请一实施例的存储设备的结构示意图；FIG. 1 is a schematic structural diagram of a storage device according to an embodiment of the present application;
图2是本申请一实施例的存储设备的存储模块的结构示意图；2 is a schematic structural diagram of a storage module of a storage device according to an embodiment of the present application;
图3是本申请一实施例的存储设备的存储模块的结构示意图；3 is a schematic structural diagram of a storage module of a storage device according to an embodiment of the present application;
图4是本申请一实施例的云端存储系统的结构以及与存储设备之间进行数据传输的示意图；4 is a schematic diagram of a structure of a cloud storage system according to an embodiment of the present application and a schematic diagram of data transmission with a storage device;
图5是本申请一实施例的云端存储系统的结构以及与存储设备之间进行数据传输的示意图；FIG. 5 is a schematic diagram of a structure of a cloud storage system and data transmission with a storage device according to an embodiment of the present application;
图6是本申请一实施例的存储设备的数据存储方法的流程示意图。FIG. 6 is a schematic flowchart of a data storage method of a storage device according to an embodiment of the present application.
具体实施方式Detailed ways
下面结合附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请一部分实施例，而非全部实施例。基于本申请中的实施例，本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。在不冲突的情况下，下述各个实施例及其技术特征可以相互组合。The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application. In the case of no conflict, the following various embodiments and their technical features can be combined with each other.
请参考图1，为本发明一实施例的存储设备的结构示意图。Please refer to FIG. 1 , which is a schematic structural diagram of a storage device according to an embodiment of the present invention.
该实施例中，所述存储设备特别是指U盘、移动硬盘、移动设备、SD卡、TF卡等，具备存储空间的便携设备。其他非便携式的存储设备也同样可以采用本发明的技术方案。In this embodiment, the storage device particularly refers to a U disk, a mobile hard disk, a mobile device, an SD card, a TF card, etc., which are portable devices with storage space. Other non-portable storage devices can also adopt the technical solutions of the present invention.
具体的，所述存储设备包括：存储模块110和连接至所述存储模块110的数据传输模块120。Specifically, the storage device includes: a
所述存储模块110，用于存储本地的数据文件以及网络地址索引文件，所述网络地址索引文件指向云端存储空间内对应的云端存储地址。所述存储模块110包括非易失性的存储器件，通常采用闪存作为存储介质，可以与终端进行数据交互，易于便携存储数据。所述云端存储地址可以是一个网络地址，包括服务器ip号等必要的信息，以便能够通过该云端存储地址直接访问到数据文件。The
所述数据传输模块120，用于在存储设备本地与云端存储系统之间进行数据传输，包括：将本地的满足预设条件的数据文件上传至云端存储系统，并形成与编辑所述云端存储地址的网络地址索引文件。The
所述数据传输模块120还用于响应对网络地址索引文件的操作，访问对应的云端存储地址和/或下载对应的云端存储地址内的数据文件。借助联网的数据终端的网络传输功能，进行数据上传或下载。所述对网络地址索引文件的操作可以通过所述存储设备自身，或者所述便携设备连接至的终端设备进行操作。在一个实施例中，所述存储设备为手机，具有存储空间浏览并操作的功能，用户可以直接通过手机访问存储空间，对地址索引文件进行浏览、移动等操作。在另一个实施例中，所述存储设备可以为U盘，可以将U盘连接至电脑，通过电脑访问U盘的存储空间，通过电脑的操作系统对U盘内的地址索引文件执行浏览、移动等操作。The
所述数据传输模块120可以利用有线或者无线方式，通过接入互联网的终端与云端存储系统之间建立数据传输路径，或者所述数据传输模块120自身具备电源以及通信功能。例如，通过接口连接已经联网的电脑，或者所述数据传输模块120具备wifi、蓝牙或红外等无线连接功能，通过无线方式，连接至电脑、手机等联网的终端设备。The
所述数据传输模块120自身还可以具备联网功能，可以通过WIFI或GSM网络直接接入互联网，与云端存储系统之间建立数据传输路径。The
请参考图2，为本发明一实施例的所述存储模块内的存储空间的示意图。Please refer to FIG. 2 , which is a schematic diagram of a storage space in the storage module according to an embodiment of the present invention.
该实施例中，所述存储模块110的存储空间被逻辑性的划分为存储区111和缓存区112。In this embodiment, the storage space of the
所述存储区111作为数据存储区域，用于存放本地的数据文件以及存储在云端的数据文件所对应的网络地址索引文件。所述数据文件和网络地址索引文件可以分别存储于连续的物理存储空间内。在一些实施例中，可以将存储区111逻辑性的划分为本地存储区域FAT和索引文件存储区域INDEX，分别用于存储本地数据文件和网络地址索引文件，便于用于整理和查询相关文件。The
该实施例中，所述存储模块110还具备缓存区112，所述缓存区112可以是物理上的固定存储区域，也可以是由数据传输模块120每次在需要进行数据传输时，临时分配的一定大小的在物理上连续或不连续的存储空间。由于所述数据传输模块120在传输数据时，容易受到带宽限制，实时数据传输的效率受限，可以预先将待上述的数据，或已下载的数据缓存在所述缓存区112内，以提高数据传输或下载的效率。例如，在需要播放云端存储的视频时，实时下载容易受到网速、带宽的限制，存在等待的现象；此时，可以在本地播放已下载的视频文件时，在后台持续不间断的下载视频流数据，并提前缓存在所述缓存区112内，后续本地播放时，就能够直接从本地的缓存区112内获取数据，提高视频播放的流畅度。所述缓存区112还可以采用比存储区111具有更块数据传输效率的存储介质，非易失或易失性的均可以，例如可以采用SRAM。In this embodiment, the
所述存储模块110内存储的网络地址索引文件包括缩略图、二维码、文本文件中的至少一种，所述网络地址索引文件编辑有超链接等信息，指向对应的数据文件在云端存储系统内的云端存储地址，可以配置为单击、双击或各种操作，访问所述云端存储地址，以进行文件查看或下载。The network address index file stored in the
在一些实施例中，所述网络地址索引文件可以为缩略图，根据对应的数据文件的不同，所述缩略图可以有各种形式，例如可以为照片的缩略图、视频文件的某一帧图形的缩略图、文件夹快捷方式图等，用户可以根据具体的情况进行配置，在此不做限定。以缩略图的形式呈现已上传的数据文件对应的索引文件，可以让用户在查看存储设备的存储内容时，在感官上还是认为文件依旧存储在本地，提高使用体验。In some embodiments, the network address index file may be a thumbnail image, and the thumbnail image may have various forms according to different corresponding data files, for example, it may be a thumbnail image of a photo, a certain frame of graphics of a video file The thumbnail image, folder shortcut map, etc., can be configured by the user according to the specific situation, which is not limited here. Presenting the index files corresponding to the uploaded data files in the form of thumbnails allows users to feel that the files are still stored locally when viewing the storage content of the storage device, improving the user experience.
所述网络地址索引文件具有可移动性，各个云端存储的数据文件的网络地址具有唯一性，可以通过将网络地址索引文件移动至其他的存储设备或终端内，在其他设备或终端访问该存储地址，进而提供数据访问的便捷性。由于网络地址索引文件的通常很小，通过转移网络地址索引文件的方式转移数据文件，可以提高数据迁移的效率。The network address index file is portable, and the network addresses of the data files stored in each cloud are unique. By moving the network address index file to other storage devices or terminals, other devices or terminals can access the storage address. , thereby providing convenience for data access. Since the network address index file is usually small, transferring the data file by transferring the network address index file can improve the efficiency of data migration.
请参考图3，为本发明另一实施例的存储设备的结构示意图。Please refer to FIG. 3 , which is a schematic structural diagram of a storage device according to another embodiment of the present invention.
该实施例钟，所述存储设备还包括数据分析模块130，连接至所述存储模块110，所述数据分析模块130根据预设条件判断是否将存储模块110内的数据文件上传至云端存储系统。所述数据分析模块130与所述数据传输模块120连接，用于在判断数据文件需要上传至云端存储系统时，向数据分析模块130输出对应的控制信号。所述预设条件包括文件存储时间超过第一阈值、文件大小超过第二阈值、距离最近一次被打开的时间超过第三阈值以及设备存储容量小于第四阈值中的至少一个。In this embodiment, the storage device further includes a
在一个实施例中，所述第一阈值可以被设置为半年或1年等，当某个数据文件在存储设备内的存储时间超过该第一阈值时，所述数据传输模块120在下次联网时，将该数据文件上传。In one embodiment, the first threshold may be set to six months or one year, etc. When the storage time of a certain data file in the storage device exceeds the first threshold, the
在另一实施例中，若某一数据文件距离上次被打开的时间已超过第三阈值，例如3个月、半年等，表示用于使用该文件的频率非常低，所述数据传输模块120在下次联网时，将该数据文件上传。In another embodiment, if the time since a certain data file was opened last time has exceeded a third threshold, such as 3 months, half a year, etc., indicating that the frequency of using the file is very low, the
在其他实施例中，也可以是数据文件满足其中至少两个条件时才进行上传，或者对各个条件设置优先级，根据各个优先级判断是否需要上传。例如，当设备存储容量小于第四阈值作为最高优先级时，将文件大小超过第二阈值，或者存储时间超过第一阈值，或者最近被打开的时间超过第三阈值的数据文件上传至云端。In other embodiments, the data file may be uploaded only when at least two of the conditions are met, or a priority may be set for each condition, and whether uploading is required is determined according to each priority. For example, when the storage capacity of the device is less than the fourth threshold as the highest priority, upload data files whose file size exceeds the second threshold, or whose storage time exceeds the first threshold, or whose most recent opening time exceeds the third threshold, to the cloud.
用户可以根据实际需求对上述预设条件进行设置，在此不作限定。The user can set the above preset conditions according to actual needs, which is not limited here.
在一些实施例中，所述数据分析模块130，还用于监测所述存储设备的存储空间大小，当监测到存储设备的剩余存储空间不足时，会启动数据清理功能，清理过程中遵循预设条件。与上文所述预设条件下的第一至第四阈值不同之处在于，前述实施例中，存储设备是静态的存储数据和上传数据到云端，此处时实或周期性的对存储空间大小进行监控，动态的存储数据到存储设备中，以及动态的将数据上传到云端中。此处的预设条件遵循最小量清理和最早数据优先清理原则，以确保保留尽量多的最新数据在存储设备中。在一些实施例中，所述存储设备可以用于行车记录仪，所述数据分析模块130优先保留预设时间(例如1小时)内的数据，在文件被上传至云端后，在存储空间不足的情况下，按照上传时间先后，优先清理最早上传成功的数据，再清理10小时前已上传的数据，如果空间仍然不足，再清理1小时前未上传的数据。In some embodiments, the
优选地，所述数据分析模块130在进行数据清理时，对已经完成上传的数据会进行删除操作，但是在删除数据的同时，会在日志系统中记录所清理的时间发生时间、类型、级别等必要数据信息，以便后续能查询到被清理的数据文件的简要信息，此举也方便到云端中查询被清理的数据文件。Preferably, when the
基于所述数据分析模块130，解决了传统的存储设备只能起到被动数据存储的作用，不能主动及时提取和分析数据在存储设备中的情况，其提高了数据在存储设备中的提取实时性和分析效率，同时增强了数据的安全性，防止数据在存储设备中的单一存储情况下的数据被篡改或删除。Based on the
进一步，本实施例中的存储设备还可以为物联网、车联网、智能家居中的存储设备，其可以通过4G/5G物联网、车联网、智能家居中的物联网通道向云端上传数据，上传过程支持断点续传，可减少网络流量的浪费以及加快数据的上传或下载。Further, the storage device in this embodiment can also be a storage device in the Internet of Things, the Internet of Vehicles, and a smart home, which can upload data to the cloud through the Internet of Things channel in the 4G/5G Internet of Things, the Internet of Vehicles, and the smart home, and upload data to the cloud. The process supports resumed uploading from a breakpoint, which can reduce the waste of network traffic and speed up the upload or download of data.
在一些实施例中，所述存储设备的数据传输模块120接收数据分析模块130处理好的数据文件，实现数据文件上传至云端。所述数据传输模块120通过互联网终端上传文件，优选地，数据分析模块130向终端提供云端的本地虚拟化映射功能，同时支持断点续传，简化数据文件的上传过程。In some embodiments, the
进一步，在一些实施例中，可以采用JMS、XMPP协议、AMQP协议、DDS协议、MQTT协议、CoAP协议、REST/HTTP或C-V2X通信协议的任意一种进行数据传输模块130与互联网终端之间的命令交互，实现双方数据的互传、数据远程召唤等功能，进一步，在传输通道上对交互数据进行加密传输，降低网络传输的安全风险。对网联终端与数据传输模块之间的通信采用RSA非对称加密算法进行一层加密握手，再为每次数据上传过程临时协商和分配二层AES对称加密密钥，然后对所有交互命令和上传的数据进行AES加密传输。由于互联网终端每次与数据传输模块120之间的交互密钥都是动态生成的，因为可以增强数据再网络上传输的安全性。Further, in some embodiments, any one of JMS, XMPP protocol, AMQP protocol, DDS protocol, MQTT protocol, CoAP protocol, REST/HTTP or C-V2X communication protocol can be used to perform the communication between the
被上传的数据文件，可以被集中或分布式存储于所述云端存储系统内，根据云端存储系统的具体存储架构进行存储。The uploaded data files can be stored in the cloud storage system in a centralized or distributed manner, and stored according to the specific storage architecture of the cloud storage system.
与所述云端存储系统的具体存储架构对应的，所述数据传输模块120在下载云端存储地址内的数据文件时，也存在多种形式，例如可以直接从云端存储系统下载集中存储的完整的数据文件，也可以时从云端存储系统下载被分布存储的多个数据块，对所述多个数据块进行数据拼接获取完整的数据文件。Corresponding to the specific storage architecture of the cloud storage system, when the
所述数据传输模块120或云端存储系统在对所述数据文件或数据块在上传或下载过程中还可以进行加/解密操作，已确保数据的安全性。The
利用分布式存储的数据块中通常可以包括：主数据以及冗余数据，在某个分布式存储节点不在线的情况下，通过其他存储节点的数据块依旧能够拼接成完整的数据文件，供下载或查阅。The data blocks using distributed storage can usually include: main data and redundant data. When a distributed storage node is offline, the data blocks of other storage nodes can still be spliced into a complete data file for download. or check.
上述实施例中所述的存储设备通过将部分数据文件上传至云端而拓展设备的存储空间，解决存储设备存储空间局限的问题。The storage device described in the above embodiment expands the storage space of the device by uploading some data files to the cloud, thereby solving the problem of limited storage space of the storage device.
本发明的实施例还提供一种适用于存储设备数据存储的云端存储系统。所述云端存储系统包括云端存储空间和云端控制模块，所述云端存储空间用于存储存储设备上传的数据文件；云端控制模块与所述云端存储空间连接，用于与存储设备之间建立数据传输连接，为所述存储设备上传的数据文件分配对应的云端存储地址，以及将所述云端存储地址反馈至所述存储设备端。Embodiments of the present invention also provide a cloud storage system suitable for storage device data storage. The cloud storage system includes a cloud storage space and a cloud control module, where the cloud storage space is used to store data files uploaded by the storage device; the cloud control module is connected to the cloud storage space for establishing data transmission with the storage device connection, assigning a corresponding cloud storage address to the data file uploaded by the storage device, and feeding back the cloud storage address to the storage device.
在一些实施例中，为了提高数据上传和下载的速度，所述数据分析模块130可以将存储设备中的每一份文件(数据)通过字符串变换算法，生成了唯一的MD5值，然后再通过数据传输模块120上传。在后续有新的文件上传需求时，将待上传文件的MD5值与云端数据库中的MD5值进行比较，确定是否有相同MD5值的文件，若所述云端数据库中不存在相同MD5值的文件，则上传所述待上传文件至所述云端数据库中，若所述云端数据库中存在相同的MD5值文件，则不上传所述待上传文件。其目的是确保文件不被重复上传以及提高用户在操作文件上传时的速度(重复的文件不会被上传，但是展示给用户的是文件快速上传成功)，节省云端数据库中的容量，提高云端数据库的数据存储利用率，每一份文件存储在云端数据库中的唯一性确保了后续数据库数据维护的方便性，毋须对数据库中的数据进行检索和删除重复文件操作。In some embodiments, in order to improve the speed of data uploading and downloading, the
请参考图4，为本发明一实施例的云端存储系统300的结构示意图，以及与存储设备100之间进行数据传输的示意图。Please refer to FIG. 4 , which is a schematic structural diagram of a cloud storage system 300 according to an embodiment of the present invention, and a schematic diagram of data transmission with the
该实施例中，所述云端存储系统300包括云端控制模块310，以及集中式存储空间320。In this embodiment, the cloud storage system 300 includes a
所述集中式存储空间320由一台或多台主计算机组成中心节点，数据集中存储于这个中心节点的存储器中，而数据的存储与控制处理完全交由所述云端控制模块310处理。The
云端控制模块310接收从存储设备100上传的数据文件，为其在所述集中式存储空间320内分配存储地址，并进行存储，以及将所述云端存储地址反馈至所述存储设备端，以便于存储设备100根据上述云端存储地址形成相应的索引文件进行保存。The
云端控制模块310还用于响应访问设备对于某个云端存储地址的访问请求，从存储空间320的对应地址内获取对应的数据文件，并传输给访问设备，所述访问设备可以是存储设备本身，也可以是获取有所述索引文件的其他设备。The
同一个存储设备100可以分别通过不同的终端设备200与所述云端存储系统300建立数据传输路径。The
请参考图5，为本发明另一实施例的云端存储系统400的结构示意图，以及与存储设备100之间进行数据传输的示意图。Please refer to FIG. 5 , which is a schematic structural diagram of a
所述云端存储系统400包括云端控制模块410以及多个分布式存储空间420。The
所述云端控制模块410还用于将所述云端存储地址映射为多个分布式存储空间的存储地址，并将上传的数据文件分割为多个数据块，分配至多个分布式存储空间420进行分布式存储。The
所述分布式的存储空间420可以是在空间上随意分布的多台计算机内的存储空间，可以由计算机自身的处理器为存储的数据块分配存储地址，并反馈至所述云端控制模块410，由所述云端控制模块410将多个分布存储空间内的数据块的存储地址，映射成一云端网络存储地址，反馈至存储设备端，以形成对应的网络地址索引文件。The distributed
所述数据块包括主数据以及冗余数据，所述冗余数据作为数据副本，在某个存储空间420未接入网络或数据丢失情况下，可以从其他存储空间420的冗余数据中读取到缺失的数据。The data block includes main data and redundant data. The redundant data is used as a data copy. In the case that a
分布式存储空间420的接入数量、接入IP均可以随时发生变化，只需要通过所述云端控制模块410与对各个所述分布式存储空间420之间建立连接即可进行存储。所述云端控制模块410由于对数据块进行分流分配至各个分布式存储空间420内。存储于各分布式存储空间420内的数据块由于并不完整，因此无法被读取查阅，具有较高的隐私性。The access quantity and access IP of the distributed
所述分布式存储空间420可以是其他用户的终端闲置存储资源，存储于各分布式存储空间420内的数据块由于并不完整，因此无法被读取查阅，具有较高的隐私性。还可以对数据块进行加密操作，进一步增强数据安全性。The distributed
在用户通过索引文件访问数据文件时，所述云端控制模块410还用于从多个分布式存储空间420内获取多个对应的数据块，直接下传至访问设备，由访问设备将数据进行拼接获取完整数据文件，或者，由云端控制模块410将多个数据块拼接为完整的数据文件后再下传至访问设备。When a user accesses a data file through an index file, the
本发明的实施例还提供一种存储设备的数据存储方法。Embodiments of the present invention also provide a data storage method of a storage device.
请参考图6，为所述存储设备的数据存储方法的流程示意图。Please refer to FIG. 6 , which is a schematic flowchart of a data storage method of the storage device.
所述数据存储方法包括如下步骤：The data storage method includes the following steps:
步骤S101:判断存储设备内本地存储的数据文件是否满足预设条件。Step S101: Determine whether the data file stored locally in the storage device satisfies a preset condition.
在一些实施例中，所述预设条件包括：文件存储时间超过第一阈值、文件大小超过第二阈值、距离最近一次被打开的时间超过第三阈值以及设备存储容量小于第四阈值中的至少一个。In some embodiments, the preset conditions include at least one of: the file storage time exceeds a first threshold, the file size exceeds a second threshold, the time since it was last opened exceeds a third threshold, and the storage capacity of the device is less than a fourth threshold. One.
步骤S102:将满足预设条件的数据文件上传至云端存储系统，并获取该数据文件在云端存储空间内的云端存储地址。Step S102: Upload the data file that meets the preset condition to the cloud storage system, and obtain the cloud storage address of the data file in the cloud storage space.
在一些实施例中，将所述数据文件集中或分布式存储于云端存储系统。In some embodiments, the data files are centrally or distributedly stored in a cloud storage system.
在一些实施例中，所述分布式存储的方法包括：将所述云端存储地址映射为多个分布式存储空间的存储地址，并将上传的数据文件分割为多个数据块，分配至多个分布式存储空间进行分布式存储。In some embodiments, the distributed storage method includes: mapping the cloud storage address to storage addresses of multiple distributed storage spaces, dividing the uploaded data file into multiple data blocks, and assigning them to multiple distributed storage spaces. type storage space for distributed storage.
在一些实施例中，还包括：将存储设备中的数据文件通过字符串变换算法，生成每一份数据文件的唯一的MD5值，并且将待上传的数据文件的MD5值与云端存储系统内的各个数据文件的MD5值进行比较，当云端存储系统内不存在相同的MD5值时，将所述待上传得数据文件上传至云端存储系统。In some embodiments, the method further includes: generating a unique MD5 value of each data file by passing the data file in the storage device through a string transformation algorithm, and comparing the MD5 value of the data file to be uploaded with the data in the cloud storage system The MD5 values of each data file are compared, and when the same MD5 value does not exist in the cloud storage system, the data file to be uploaded is uploaded to the cloud storage system.
步骤S103:在设备本地存储与上传的数据文件对应的网络地址索引文件，所述网络地址索引文件指向该数据文件的云端存储地址。Step S103: Store a network address index file corresponding to the uploaded data file locally on the device, and the network address index file points to the cloud storage address of the data file.
在一些实施例中，所述网络地址索引文件包括编辑有超链接的缩略图、二维码、文本中的至少一种。In some embodiments, the network address index file includes at least one of thumbnails, two-dimensional codes, and texts edited with hyperlinks.
在一些实施例中，在所述数据传输模块进行数据传输时，对待传输数据在本地进行缓存。In some embodiments, when the data transmission module performs data transmission, the data to be transmitted is locally cached.
在一些实施例中，所述数据存储方法还包括：响应对网络地址索引文件的操作，访问对应的云端存储地址和/或下载对应的云端存储地址内的文件。In some embodiments, the data storage method further includes: in response to an operation on the network address index file, accessing a corresponding cloud storage address and/or downloading a file in the corresponding cloud storage address.
在一些实施例中，下载对应的云端存储地址内的文件的方法包括：下载获取多个分布式存储的数据块后，对所述多个数据块进行数据拼接，以获取完整的数据文件。In some embodiments, the method for downloading a file in a corresponding cloud storage address includes: after downloading and obtaining a plurality of distributed storage data blocks, data splicing is performed on the plurality of data blocks to obtain a complete data file.
在一些实施例中，还包括：监测所述存储设备的存储空间大小，当监测到剩余存储空间小于预设值时，遵循预设条件对存储空间内的数据进行清理。In some embodiments, the method further includes: monitoring the size of the storage space of the storage device, and when it is detected that the remaining storage space is less than a preset value, cleaning the data in the storage space according to preset conditions.
在一些实施例中，所述数据分析模块还用于在对存储空间内的数据进行清理的同时在日志系统记录数据清理相关的操作信息。In some embodiments, the data analysis module is further configured to record operation information related to data cleaning in the log system while cleaning the data in the storage space.
上述前述关于存储设备以及云端存储系统的实施例的内容均援引在此，相关特征的描述不再赘述。The above-mentioned contents of the embodiments of the storage device and the cloud storage system are all cited here, and the description of the relevant features will not be repeated.
本发明的实施例，还提供一种存储设备，包括：处理器；存储器，存储有能够被所述处理器执行的计算机程序，所述计算机程序被执行时能够执行上述实施例中所述的数据存储方法。An embodiment of the present invention further provides a storage device, comprising: a processor; and a memory, storing a computer program that can be executed by the processor, and when executed, the computer program can execute the data described in the above embodiments storage method.
以上所述仅为本申请的实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，例如各实施例之间技术特征的相互结合，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。The above descriptions are only the embodiments of the present application, and are not intended to limit the scope of the patent of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present application, such as the interaction of technical features between the embodiments Combination, or direct or indirect application in other related technical fields, are equally included in the scope of patent protection of this application.
Claims (16)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110298190.XACN114610679A (en) | 2021-03-19 | 2021-03-19 | Storage device, data storage method thereof and cloud storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110298190.XACN114610679A (en) | 2021-03-19 | 2021-03-19 | Storage device, data storage method thereof and cloud storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114610679Atrue CN114610679A (en) | 2022-06-10 |
Family
ID=81857380
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110298190.XAPendingCN114610679A (en) | 2021-03-19 | 2021-03-19 | Storage device, data storage method thereof and cloud storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114610679A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116595007A (en)* | 2023-05-23 | 2023-08-15 | 建材广州工程勘测院有限公司 | Geotechnical engineering geological data management system |
| CN118283030A (en)* | 2024-05-13 | 2024-07-02 | 重庆数智融合创新科技有限公司 | File transmission method and system based on Internet of things and cloud |
| WO2024140002A1 (en)* | 2022-12-26 | 2024-07-04 | 花瓣云科技有限公司 | Storage space management method and apparatus, electronic device, and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102333079A (en)* | 2011-02-25 | 2012-01-25 | 北京兴宇中科科技开发股份有限公司 | Method for clearing disk space |
| CN103237054A (en)* | 2013-03-28 | 2013-08-07 | 东莞宇龙通信科技有限公司 | Photo/video storage method and system, and mobile terminal |
| CN103384257A (en)* | 2012-05-04 | 2013-11-06 | 华为终端有限公司 | Method for accessing network storage and network access device |
| CN103577503A (en)* | 2012-08-10 | 2014-02-12 | 鸿富锦精密工业（深圳）有限公司 | Cloud file storage system and method |
| CN106850837A (en)* | 2017-03-06 | 2017-06-13 | 苏州汉辰数字科技有限公司 | A kind of cloud storage quantal data and processing system |
| CN109451004A (en)* | 2018-10-30 | 2019-03-08 | 四川乐为科技有限公司 | Cloud storage system and method based on cloud bridge |
| CN109981702A (en)* | 2017-12-27 | 2019-07-05 | 深圳市优必选科技有限公司 | File storage method and system |
| CN110650164A (en)* | 2018-06-26 | 2020-01-03 | 马上消费金融股份有限公司 | File uploading method and device, terminal and computer storage medium |
- 2021
- 2021-03-19CNCN202110298190.XApatent/CN114610679A/enactivePending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102333079A (en)* | 2011-02-25 | 2012-01-25 | 北京兴宇中科科技开发股份有限公司 | Method for clearing disk space |
| CN103384257A (en)* | 2012-05-04 | 2013-11-06 | 华为终端有限公司 | Method for accessing network storage and network access device |
| CN103577503A (en)* | 2012-08-10 | 2014-02-12 | 鸿富锦精密工业（深圳）有限公司 | Cloud file storage system and method |
| CN103237054A (en)* | 2013-03-28 | 2013-08-07 | 东莞宇龙通信科技有限公司 | Photo/video storage method and system, and mobile terminal |
| CN106850837A (en)* | 2017-03-06 | 2017-06-13 | 苏州汉辰数字科技有限公司 | A kind of cloud storage quantal data and processing system |
| CN109981702A (en)* | 2017-12-27 | 2019-07-05 | 深圳市优必选科技有限公司 | File storage method and system |
| CN110650164A (en)* | 2018-06-26 | 2020-01-03 | 马上消费金融股份有限公司 | File uploading method and device, terminal and computer storage medium |
| CN109451004A (en)* | 2018-10-30 | 2019-03-08 | 四川乐为科技有限公司 | Cloud storage system and method based on cloud bridge |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024140002A1 (en)* | 2022-12-26 | 2024-07-04 | 花瓣云科技有限公司 | Storage space management method and apparatus, electronic device, and storage medium |
| CN116595007A (en)* | 2023-05-23 | 2023-08-15 | 建材广州工程勘测院有限公司 | Geotechnical engineering geological data management system |
| CN118283030A (en)* | 2024-05-13 | 2024-07-02 | 重庆数智融合创新科技有限公司 | File transmission method and system based on Internet of things and cloud |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11556388B2 (en) | Frozen indices | |
| US9792344B2 (en) | Asynchronous namespace maintenance | |
| US11531491B2 (en) | Systems and methods for random to sequential storage mapping | |
| US8650159B1 (en) | Systems and methods for managing data in cloud storage using deduplication techniques | |
| CN110401724B (en) | File management method, file transfer protocol server and storage medium | |
| TWI706315B (en) | Storage device | |
| CN114610679A (en) | Storage device, data storage method thereof and cloud storage system | |
| US20120110005A1 (en) | System and method for sharing online storage services among multiple users | |
| CN110958300B (en) | A method, system, apparatus, electronic device and computer-readable medium for uploading data | |
| CN111339192A (en) | Distributed edge computing data storage system | |
| US20180213025A1 (en) | Transparent deferred spooling store and forward based on standard network system and client interface | |
| US10503693B1 (en) | Method and system for parallel file operation in distributed data storage system with mixed types of storage media | |
| US20150350371A1 (en) | Caching and syncing mechanisms for a cloud library | |
| US11416176B2 (en) | Function processing using storage controllers for load sharing | |
| WO2017206754A1 (en) | Storage method and storage device for distributed file system | |
| CN102480508B (en) | Multi-user shared network storage service system and method thereof | |
| US20180248977A1 (en) | Selective distribution of messages in a publish-subscribe system | |
| WO2023093608A1 (en) | Automatic distributed cloud storage scheduling interaction method and apparatus, and device | |
| CN107615259A (en) | A kind of data processing method and system | |
| EP3555767B1 (en) | Partial storage of large files in distinct storage systems | |
| US20090327303A1 (en) | Intelligent allocation of file server resources | |
| CN119484512A (en) | File transmission and storage method, device, computer equipment, readable storage medium and program product | |
| KR101470857B1 (en) | Network distributed file system and method using iSCSI storage system | |
| CN117311596A (en) | Data reading method, data storage method, device and system | |
| CN115905115A (en) | File storage method, reading method and device, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |