技术领域technical field
本发明属于云存储领域，涉及一种用户空间文件系统实现方法，具体涉及一种面向MongoDB的用户空间文件系统实现方法。The invention belongs to the field of cloud storage, and relates to a user space file system implementation method, in particular to a MongoDB-oriented user space file system implementation method.
背景技术Background technique
在大数据的时代背景下，存储问题重新成为学术界和产业界的焦点，传统的技术和方法已经很难应对由数据量的剧增而带来的新的挑战。云存储正是在这一背景下在云计算概念上延伸和发展出来的一个新的概念，指通过集群应用、网格技术或分布式文件系统等功能，将网络中大量各种不同类型的存储设备通过应用软件集合起来协同工作，共同对外提供数据存储和业务访问功能的一个系统。在云端，各种存储设备通过应用软件集成管理起来(以MongoDB集群服务器为例)，用户使用终端(PC、手机等)通过互联网访问云端提供的存储服务。In the context of the era of big data, storage issues have become the focus of academia and industry again, and traditional technologies and methods have been difficult to cope with the new challenges brought about by the sharp increase in data volume. Cloud storage is a new concept extended and developed from the concept of cloud computing under this background. A system in which devices work together through application software to jointly provide data storage and business access functions. In the cloud, various storage devices are integrated and managed through application software (take the MongoDB cluster server as an example), and users use terminals (PCs, mobile phones, etc.) to access the storage services provided by the cloud through the Internet.
用户空间文件系统(Filesystem in Userspace，简称FUSE)指完全在用户态实现的文件系统。它方便开发人员在不触及内核的前提下轻松定制新的文件系统。User space file system (Filesystem in Userspace, referred to as FUSE) refers to a file system completely implemented in user mode. It allows developers to easily customize new file systems without touching the kernel.
在产业界，基于FUSE实现的用户空间文件系统已有很多，比较知名的有ExpanDrive、GlusterFS、SSHFS等。In the industry, there are already many user space file systems implemented based on FUSE, among which ExpanDrive, GlusterFS, SSHFS, etc. are more well-known.
ExpanDrive：是一款很实用的网盘和云服务储存文件管理软件，可以让用户本地化管理网盘或云存储中的文件，让文件修改、下载和删除等操作变得非常简单。ExpanDrive: It is a very practical network disk and cloud service storage file management software, which allows users to locally manage files in the network disk or cloud storage, making file modification, download and deletion very simple.
GlusterFS：是一个大尺度文件系统。它是各种不同的存储服务器之上的组合，这些服务器由以太网或无限带宽技术Infiniband以及远程直接内存访问RDMA互相融汇，最终所形成的一个大的并行文件系统网络。GlusterFS: It is a large-scale file system. It is a combination of various storage servers, these servers are integrated by Ethernet or Infiniband and remote direct memory access RDMA, and finally form a large parallel file system network.
SSHFS：可以把SSH连接到的主机资源，映射到本机的文件系统当中，然后用户可以像操作本地文件一样进行操作，而实际的文件改动将通过SSH传输到远程主机当中。SSHFS: It can map the host resources connected to SSH to the local file system, and then users can operate like local files, and the actual file changes will be transmitted to the remote host through SSH.
在学术界，相关文献有文献1及文献2，其中，文献1为邓宏涛在江汉大学学报，Vol.38 No.2，Jun.2010发表的关系数据库中树型结构信息的处理方法研究；文献2为汪建，方洪鹰，陈昌川在重庆师范大学学报，Vol.24 No.4，Oct.2007发表的一种改进的基于数据库的树存储策略。文献1介绍了树型结构的概念与特点，讨论了数据库中树形结构信息的关系表结构设计方案，比较了关系数据库中树型结构信息的多种处理方法，给出了采用自关联映射的表结构设计，并结合递归CTE技术探讨了关系数据库中树型结构信息的具体实现方法。文献2讨论了如何在关系数据库管理系统中存放压缩的一般树，并在维护海量数据的同时，降低数据冗余。本发明方法针对上述文献对支持非关系型数据库的不足之处，提出了在非关系型数据库MongoDB中存储树的方法。In academia, related literature includes literature 1 and literature 2, among which, literature 1 is the research on the processing method of tree structure information in relational database published by Deng Hongtao in Journal of Jianghan University, Vol.38 No.2, Jun.2010; literature 2 An improved database-based tree storage strategy published for Wang Jian, Fang Hongying, and Chen Changchuan in the Journal of Chongqing Normal University, Vol.24 No.4, Oct.2007. Document 1 introduces the concept and characteristics of tree structure, discusses the design scheme of relational table structure for tree structure information in database, compares various processing methods of tree structure information in relational database, and gives the method of using self-association mapping Table structure design, combined with recursive CTE technology, discusses the specific implementation method of tree structure information in relational database. Document 2 discusses how to store compressed general trees in a relational database management system and reduce data redundancy while maintaining massive data. The method of the present invention proposes a method for storing trees in the non-relational database MongoDB aiming at the deficiencies of the above-mentioned documents in supporting non-relational databases.
GridFS-Fuse是将云端MongoDB GridFS数据库实例挂载至本地的一个用户态文件系统。2009年Paul Rosania对其进行尝试的开发，在所公布的发布版说明中指出GridFS-Fuse当前的未解决问题为：不支持目录操作以及读写效率待验证。GridFS-Fuse is a user mode file system that mounts the cloud MongoDB GridFS database instance to the local. In 2009, Paul Rosania tried to develop it, and pointed out in the published release notes that the current unresolved problems of GridFS-Fuse are: it does not support directory operations and the read and write efficiency is yet to be verified.
发明内容Contents of the invention
本发明的目的在于克服上述现有技术的缺点，提供了一种面向MongoDB的用户空间文件系统实现方法，该实现方法支持目录操作，并且具有占用空间少，读写效率高等优点。The purpose of the present invention is to overcome the shortcoming of above-mentioned prior art, provide a kind of user space file system implementation method facing MongoDB, this implementation method supports catalog operation, and has advantages such as less occupied space, high reading and writing efficiency.
为达到上述目的，本发明所述的面向MongoDB的用户空间文件系统实现方法面向MongoDB的用户空间文件系统实现方法，其特征在于，包括以下步骤：构建用于储存文件目录结构信息以及节点元数据信息的fs.nodes集合，再利用MongoDB GridFS存储文件，同时在利用MongoDBGridFS存储文件的过程中自动生成用于存储文件元数据信息的fs.files集合及用于存储文件数据信息的fs.chunks集合，Linux文件系统与MongoDB数据库实现节点的创建及删除过程中通过fs.nodes集合、fs.files集合及fs.chunks集合关联，具体的，目录节点与fs.nodes集合相互关联，文件节点与fs.nodes集合、fs.files集合以及fs.chunks集合相互关联，节点与节点之间的父子关系由fs.nodes集合的链接结构所决定。In order to achieve the above object, the MongoDB-oriented user space file system implementation method of the present invention is characterized in that it includes the following steps: constructing a directory structure information and node metadata information for storing files The fs.nodes collection of MongoDB GridFS is used to store files, and the fs.files collection for storing file metadata information and the fs.chunks collection for storing file data information are automatically generated during the process of using MongoDB GridFS to store files. Linux The file system and the MongoDB database are associated with the fs.nodes collection, fs.files collection, and fs.chunks collection during the creation and deletion of nodes. Specifically, directory nodes are associated with fs.nodes collections, and file nodes are associated with fs.nodes collections. , fs.files collection, and fs.chunks collection are interrelated, and the parent-child relationship between nodes is determined by the link structure of fs.nodes collection.
所述fs.nodes集合中的各个字段的含义如下：The meanings of the fields in the fs.nodes collection are as follows:
_id字段：用于记录节点的全局唯一标识ID，类型为Object_id，由mongoDB默认自动赋值；_id field: used to record the globally unique ID of the node, the type is Object_id, and is automatically assigned by default by mongoDB;
name字段：用于记录节点的名称，类型为字符串；name field: used to record the name of the node, the type is a string;
type字段：用于记录节点的类型，类型为整型；type field: used to record the type of the node, the type is an integer;
depth字段：用于记录节点所在树结构的深度，类型为整型；depth field: used to record the depth of the tree structure where the node is located, the type is an integer;
abs_path字段：用于记录节点的绝对路径名称，即为根节点到当前节点的路径，类型为字符串；abs_path field: used to record the absolute path name of the node, which is the path from the root node to the current node, and the type is a string;
parent_id：用于记录节点的父节点ID，类型为Object_id；parent_id: used to record the ID of the parent node of the node, the type is Object_id;
meta_data：用于记录节点的元数据信息，类型为BSON；meta_data: used to record the metadata information of the node, the type is BSON;
其中，meta_data各个字段含义如下：Among them, the meanings of each field of meta_data are as follows:
file_id字段：用于记录文件ID，类型为Object_id，其中，当节点是文件，则file_id字段有效；当节点为目录时，则file_id字段无效；file_id field: used to record the file ID, the type is Object_id, where, when the node is a file, the file_id field is valid; when the node is a directory, the file_id field is invalid;
mode字段：用于记录节点访问模式，类型为整型；mode field: used to record the node access mode, the type is integer;
uid字段：用于记录用户id，类型为整型；uid field: used to record user id, the type is integer;
gid字段：用于记录组id，类型为整型；gid field: used to record the group id, the type is integer;
nlink字段：用于记录节点连接数，类型为整型，当节点为文件时，则nlink字段为该节点的连接数；当节点为目录时，则nlink字段为该目录下所包含的子目录数；nlink field: used to record the number of node connections, the type is integer, when the node is a file, the nlink field is the number of connections of the node; when the node is a directory, the nlink field is the number of subdirectories contained in the directory ;
atime字段：用于记录节点上一次存取时间；atime field: used to record the last access time of the node;
mtime字段：用于记录节点上一次修改时间；mtime field: used to record the last modification time of the node;
ctime字段：用于记录上一次状态修改时间。ctime field: used to record the last state modification time.
所述节点的创建包括文件节点的创建及目录节点的创建。The creation of the node includes the creation of the file node and the creation of the directory node.
所述文件节点的创建包括以下步骤：The creation of the file node includes the following steps:
检查所需创建的文件节点是否为空文件节点，当所述文件节点为空文件节点时，则关闭MongoDB数据库，当所述文件节点不为空文件节点时，则将Linux文件系统连接MongoDB数据库，并根据所需创建的文件节点的绝对路径名称在MongoDB数据库中查找是否已存在该文件节点，当在MongoDB数据库中已存在该文件节点时，则关闭MongoDB数据库，当在MongoDB数据库中不存在该文件节点时，则创建MongoDBGridFS对象，通过所述MongoDB GridFS对象将文件节点的内容写入MongoDB数据库中，并获取文件ID，当没有成功获取文件ID，则直接关闭MongoDB数据库，当成功获取文件ID，则根据当前绝对路径名称计算父节点的绝对路径名，并通过父节点的绝对路径名查询获取父节点ID，当不能通过父节点的绝对路径名查询获取父节点ID，则直接关闭MongoDB数据库，当成功通过父节点的绝对路径名查询获取父节点ID，则填充fs.nodes集合中各字段的值，并将该节点插入MongoDB数据库中，当成功将该节点插入MongoDB数据库中时，则完成文件节点的创建，当不能将该节点插入MongoDB数据库中时，则直接关闭MongoDB数据库。Check whether the file node to be created is an empty file node. When the file node is an empty file node, the MongoDB database is closed. When the file node is not an empty file node, the Linux file system is connected to the MongoDB database. And according to the absolute path name of the file node to be created, find whether the file node already exists in the MongoDB database. When the file node already exists in the MongoDB database, close the MongoDB database. When the file does not exist in the MongoDB database node, then create a MongoDBGridFS object, write the content of the file node in the MongoDB database through the MongoDB GridFS object, and obtain the file ID, when the file ID is not successfully obtained, then directly close the MongoDB database, when the file ID is successfully obtained, then Calculate the absolute path name of the parent node according to the current absolute path name, and obtain the parent node ID through the absolute path name query of the parent node. If the parent node ID cannot be obtained through the absolute path name query of the parent node, the MongoDB database will be closed directly. When it succeeds Obtain the ID of the parent node through the absolute path name query of the parent node, fill in the values of each field in the fs.nodes collection, and insert the node into the MongoDB database. When the node is successfully inserted into the MongoDB database, the file node is completed. Create, when the node cannot be inserted into the MongoDB database, the MongoDB database will be closed directly.
所述目录节点的创建包括以下步骤：The creation of the directory node includes the following steps:
根据所需创建的目录节点的绝对路径名称查找是否已存在该目录节点，当存在该目录节点时，则关闭MongoDB数据库，当不存在该目录节点时，则根据当前绝对路径名称计算父节点的绝对路径名，当父节点为根目录时，则将当前目录节点的父节点ID设置为NULL，然后填充当前节点fs.nodes集合中各字段的值，再将将该目录节点插入MongoDB数据库中；Find out whether the directory node already exists according to the absolute path name of the directory node to be created. If the directory node exists, close the MongoDB database. If the directory node does not exist, calculate the absolute value of the parent node based on the current absolute path name. Path name, when the parent node is the root directory, set the parent node ID of the current directory node to NULL, then fill in the values of each field in the fs.nodes collection of the current node, and then insert the directory node into the MongoDB database;
当父节点不为根目录时，则通过父节点的绝对路径名查询获取父节点ID，并修改父节点中连接数字段的值，当不能修改父节点中连接数字段的值时，则关闭MongoDB数据库，当成功修改父节点中连接数字段的值时，则将当前目录节点的父节点ID设置为NULL，然后填充当前节点fs.nodes集合中各字段的值，再将将该目录节点插入MongoDB数据库中。When the parent node is not the root directory, query the parent node ID through the absolute path name of the parent node, and modify the value of the connection number field in the parent node. When the value of the connection number field in the parent node cannot be modified, close MongoDB Database, when the value of the connection number field in the parent node is successfully modified, the parent node ID of the current directory node is set to NULL, and then the values of each field in the fs.nodes collection of the current node are filled, and then the directory node is inserted into MongoDB in the database.
所述节点的删除包括文件节点的删除及目录节点的删除。The deletion of the node includes the deletion of the file node and the deletion of the directory node.
所述文件节点的删除包括以下步骤：The deletion of the file node includes the following steps:
将Linux文件系统连接MongoDB数据库，根据所需删除的文件节点的绝对路径名称查找是否已存在该文件节点，当不存在该文件节点时，则关闭MongoDB数据库，当存在该文件节点时，则从fs.nodes集合中删除该文件节点，当不能从fs.nodes集合中删除该文件节点时，则直接关闭MongoDB数据库，当成功从fs.nodes集合中删除该文件节点时，则创建MongoDB GridFS对象，通过该MongoDB GridFS对象从MongoDB数据库中删除该文件节点所对应的文件信息。Connect the Linux file system to the MongoDB database, and check whether the file node already exists according to the absolute path name of the file node to be deleted. If the file node does not exist, the MongoDB database will be closed. If the file node exists, it will be deleted from fs Delete the file node from the .nodes collection. When the file node cannot be deleted from the fs.nodes collection, the MongoDB database will be closed directly. When the file node is successfully deleted from the fs.nodes collection, a MongoDB GridFS object will be created. Pass The MongoDB GridFS object deletes the file information corresponding to the file node from the MongoDB database.
所述目录节点的删除包括以下步骤：The deletion of the directory node includes the following steps:
将Linux文件系统连接MongoDB数据库，根据所需删除的目录节点的绝对路径名称查找是否已存在该目录节点。当不存在该目录节点，则关闭MongoDB数据库，当存在该目录节点时，则计算该目录节点的父节点绝对路径名称，当该目录节点的父节点绝对路径名称为根目录时，则关闭MongoDB数据库，当目录节点的父节点绝对路径名称不为根目录时，则修改父节点中连接数字段的值，当不能修改父节点中连接数字段的值时，则关闭MongoDB数据库，当成功修改父节点中连接数字段的值时，则针对当前节点的所有子节点，依据子节点类型，递归调用删除文件节点中的Step 1-Step 5或递归调用删除目录节点中的Step 1-Step6。Connect the Linux file system to the MongoDB database, and check whether the directory node already exists according to the absolute path name of the directory node to be deleted. When the directory node does not exist, the MongoDB database is closed. When the directory node exists, the absolute path name of the parent node of the directory node is calculated. When the absolute path name of the parent node of the directory node is the root directory, the MongoDB database is closed. , when the absolute path name of the parent node of the directory node is not the root directory, modify the value of the connection number field in the parent node, and close the MongoDB database when the value of the connection number field in the parent node cannot be modified, and when the parent node is successfully modified When the value of the connection number field is set, for all child nodes of the current node, recursively call Step 1-Step 5 in the delete file node or recursively call Step 1-Step 6 in the delete directory node according to the child node type.
本发明具有以下有益效果：The present invention has the following beneficial effects:
本发明所述的面向MongoDB的用户空间文件系统实现方法通过目录节点与fs.nodes集合相互关联，文件节点与fs.nodes集合、fs.files集合以及fs.chunks集合元素相互关联，节点与节点之间的父子关系由fs.nodes的链接结构所决定，从而实现节点的删除及创建，并且占用存储空间少，查询效率高，通过fs.nodes集合不仅记录了节点间的关联关系，同时也记录了节点自身的元数据信息，与分别对节点间关联关系和对节点元数据设计表相比，节省了连接字段所占用的存储空间，针对文件系统特有的按路径逐级查找文件或目录的特点，该结构又能很好地节省读取查询中间节点元数据信息的时间，从而提高查询效率。The MongoDB-oriented user space file system implementation method of the present invention is associated with the fs.nodes set through the directory node, the file node is associated with the fs.nodes set, the fs.files set, and the fs.chunks set element. The parent-child relationship between is determined by the link structure of fs.nodes, so as to realize the deletion and creation of nodes, and occupy less storage space, and the query efficiency is high. The fs.nodes collection not only records the association relationship between nodes, but also records The metadata information of the node itself saves the storage space occupied by the connection field compared with the association relationship between nodes and the metadata design table for nodes respectively. For the unique feature of the file system to search for files or directories step by step according to the path, This structure can also save the time of reading and querying metadata information of intermediate nodes, thereby improving query efficiency.
附图说明Description of drawings
图1为本发明中创建及删除节点的过程中的对应关系图；Fig. 1 is the correspondence diagram in the process of creating and deleting nodes among the present invention;
图2为本发明中创建文件节点的流程图；Fig. 2 is the flowchart of creating file node among the present invention;
图3为本发明创建目录节点的流程图；Fig. 3 is the flow chart that the present invention creates directory node;
图4为本发明删除文件节点的流程图；Fig. 4 is the flowchart of deleting file node in the present invention;
图5为本发明删除目录节点的流程图。Fig. 5 is a flow chart of deleting directory nodes in the present invention.
具体实施方式Detailed ways
下面结合附图对本发明做进一步详细描述：The present invention is described in further detail below in conjunction with accompanying drawing:
参考图1，本发明所述的面向MongoDB的用户空间文件系统实现方法包括以下步骤：构建用于储存文件目录结构信息以及节点元数据信息的fs.nodes集合，再利用MongoDB GridFS存储文件，同时在利用MongoDB GridFS存储文件的过程中自动生成用于存储文件元数据信息的fs.files集合及用于存储文件数据信息的fs.chunks集合，Linux文件系统与MongoDB数据库实现节点的创建及删除过程中通过fs.nodes集合、fs.files集合及fs.chunks集合关联，具体的，目录节点与fs.nodes集合相互关联，文件节点与fs.nodes集合、fs.files集合以及fs.chunks集合元素相互关联，节点与节点之间的父子关系由fs.nodes集合内元素的链接结构所决定。With reference to Fig. 1, the MongoDB-oriented user space file system implementation method of the present invention comprises the following steps: constructing the fs. In the process of using MongoDB GridFS to store files, the fs.files collection for storing file metadata information and the fs.chunks collection for storing file data information are automatically generated, and the Linux file system and MongoDB database realize the node creation and deletion process through The fs.nodes collection, fs.files collection, and fs.chunks collection are associated. Specifically, directory nodes are associated with fs.nodes collections, and file nodes are associated with fs.nodes collections, fs.files collections, and fs.chunks collection elements. The parent-child relationship between nodes is determined by the link structure of the elements in the fs.nodes collection.
树型文件目录结构在MongoDB数据库中的映射，MongoDB GridFS存储文件时生成fs.files集合及fs.chunks集合，分别用于存储文件元数据信息和文件数据信息，其中fs.files集合及fs.chunks集合内各字段的定义分别如表1及表2所述：The mapping of the tree-type file directory structure in the MongoDB database. When MongoDB GridFS stores files, it generates fs.files collection and fs.chunks collection, which are used to store file metadata information and file data information respectively. Among them, fs.files collection and fs.chunks The definition of each field in the set is as described in Table 1 and Table 2 respectively:
表1Table 1
表2Table 2
所述fs.nodes集合中的各个字段的含义如下：The meanings of the fields in the fs.nodes collection are as follows:
_id字段：用于记录节点的全局唯一标识ID，类型为Object_id，由mongoDB默认自动赋值；_id field: used to record the globally unique ID of the node, the type is Object_id, and is automatically assigned by default by mongoDB;
name字段：用于记录节点的名称，类型为字符串；name field: used to record the name of the node, the type is a string;
type字段：用于记录节点的类型，类型为整型；type field: used to record the type of the node, the type is an integer;
depth字段：用于记录节点所在树结构的深度，类型为整型；depth field: used to record the depth of the tree structure where the node is located, the type is an integer;
abs_path字段：用于记录节点的绝对路径名称，即为根节点到当前节点的路径，类型为字符串；abs_path field: used to record the absolute path name of the node, which is the path from the root node to the current node, and the type is a string;
parent_id：用于记录节点的父节点ID，类型为Object_id；parent_id: used to record the ID of the parent node of the node, the type is Object_id;
meta_data：用于记录节点的元数据信息，类型为BSON；meta_data: used to record the metadata information of the node, the type is BSON;
其中，meta_data各个字段含义如下：Among them, the meanings of each field of meta_data are as follows:
file_id字段：用于记录文件ID，类型为Object_id，其中，当节点是文件，则file_id字段有效；当节点为目录时，则file_id字段无效；file_id field: used to record the file ID, the type is Object_id, where, when the node is a file, the file_id field is valid; when the node is a directory, the file_id field is invalid;
mode字段：用于记录节点访问模式，类型为整型；mode field: used to record the node access mode, the type is integer;
uid字段：用于记录用户id，类型为整型；uid field: used to record user id, the type is integer;
gid字段：用于记录组id，类型为整型；gid field: used to record the group id, the type is integer;
nlink字段：用于记录节点连接数，类型为整型，当节点为文件时，则nlink字段为该节点的连接数；当节点为目录时，则nlink字段为该目录下所包含的子目录数；nlink field: used to record the number of node connections, the type is integer, when the node is a file, the nlink field is the number of connections of the node; when the node is a directory, the nlink field is the number of subdirectories contained in the directory ;
atime字段：用于记录节点上一次存取时间；atime field: used to record the last access time of the node;
mtime字段：用于记录节点上一次修改时间；mtime field: used to record the last modification time of the node;
ctime字段：用于记录上一次状态修改时间。ctime field: used to record the last state modification time.
所述节点的创建包括文件节点的创建及目录节点的创建。The creation of the node includes the creation of the file node and the creation of the directory node.
参考图2，所述文件节点的创建包括以下步骤：With reference to Fig. 2, the establishment of described file node comprises the following steps:
检查所需创建的文件节点是否为空文件节点，当所述文件节点为空文件节点时，则关闭MongoDB数据库，当所述文件节点不为空文件节点时，则将Linux文件系统连接MongoDB数据库，并根据所需创建的文件节点的绝对路径名称在MongoDB数据库中查找是否已存在该文件节点，当在MongoDB数据库中已存在该文件节点时，则关闭MongoDB数据库，当在MongoDB数据库中不存在该文件节点时，则创建MongoDBGridFS对象，通过所述MongoDB GridFS对象将文件节点的内容写入MongoDB数据库中，并获取文件ID，当没有成功获取文件ID，则直接关闭MongoDB数据库，当成功获取文件ID，则根据当前绝对路径名称计算父节点的绝对路径名，并通过父节点的绝对路径名查询获取父节点ID，当不能通过父节点的绝对路径名查询获取父节点ID，则直接关闭MongoDB数据库，当成功通过父节点的绝对路径名查询获取父节点ID，则填充fs.nodes集合中各字段的值，并将该节点插入MongoDB数据库中，当成功将该节点插入MongoDB数据库中时，则完成文件节点的创建，当不能将该节点插入MongoDB数据库中时，则直接关闭MongoDB数据库。Check whether the file node to be created is an empty file node. When the file node is an empty file node, the MongoDB database is closed. When the file node is not an empty file node, the Linux file system is connected to the MongoDB database. And according to the absolute path name of the file node to be created, find whether the file node already exists in the MongoDB database. When the file node already exists in the MongoDB database, close the MongoDB database. When the file does not exist in the MongoDB database node, then create a MongoDBGridFS object, write the content of the file node in the MongoDB database through the MongoDB GridFS object, and obtain the file ID, when the file ID is not successfully obtained, then directly close the MongoDB database, when the file ID is successfully obtained, then Calculate the absolute path name of the parent node according to the current absolute path name, and obtain the parent node ID through the absolute path name query of the parent node. If the parent node ID cannot be obtained through the absolute path name query of the parent node, the MongoDB database will be closed directly. When it succeeds Obtain the ID of the parent node through the absolute path name query of the parent node, fill in the values of each field in the fs.nodes collection, and insert the node into the MongoDB database. When the node is successfully inserted into the MongoDB database, the file node is completed. Create, when the node cannot be inserted into the MongoDB database, the MongoDB database will be closed directly.
参考图3，所述目录节点的创建包括以下步骤：Referring to Figure 3, the creation of the directory node includes the following steps:
根据所需创建的目录节点的绝对路径名称查找是否已存在该目录节点，当存在该目录节点时，则关闭MongoDB数据库，当不存在该目录节点时，则根据当前绝对路径名称计算父节点的绝对路径名，当父节点为根目录时，则将当前目录节点的父节点ID设置为NULL，然后填充当前节点fs.nodes集合中各字段的值，再将将该目录节点插入MongoDB数据库中；Find out whether the directory node already exists according to the absolute path name of the directory node to be created. If the directory node exists, close the MongoDB database. If the directory node does not exist, calculate the absolute value of the parent node based on the current absolute path name. Path name, when the parent node is the root directory, set the parent node ID of the current directory node to NULL, then fill in the values of each field in the fs.nodes collection of the current node, and then insert the directory node into the MongoDB database;
当父节点不为根目录时，则通过父节点的绝对路径名查询获取父节点ID，并修改父节点中连接数字段的值，当不能修改父节点中连接数字段的值时，则关闭MongoDB数据库，当成功修改父节点中连接数字段的值时，则将当前目录节点的父节点ID设置为NULL，然后填充当前节点fs.nodes集合中各字段的值，再将将该目录节点插入MongoDB数据库中。When the parent node is not the root directory, query the parent node ID through the absolute path name of the parent node, and modify the value of the connection number field in the parent node. When the value of the connection number field in the parent node cannot be modified, close MongoDB Database, when the value of the connection number field in the parent node is successfully modified, the parent node ID of the current directory node is set to NULL, and then the values of each field in the fs.nodes collection of the current node are filled, and then the directory node is inserted into MongoDB in the database.
所述节点的删除包括文件节点的删除及目录节点的删除。The deletion of the node includes the deletion of the file node and the deletion of the directory node.
参考图4，所述文件节点的删除包括以下步骤：With reference to Fig. 4, the deletion of described file node comprises the following steps:
将Linux文件系统连接MongoDB数据库，根据所需删除的文件节点的绝对路径名称查找是否已存在该文件节点，当不存在该文件节点时，则关闭MongoDB数据库，当存在该文件节点时，则从fs.nodes集合中删除该文件节点，当不能从fs.nodes集合中删除该文件节点时，则直接关闭MongoDB数据库，当成功从fs.nodes集合中删除该文件节点时，则创建MongoDB GridFS对象，通过该MongoDB GridFS对象从MongoDB数据库中删除该文件节点所对应的文件信息。Connect the Linux file system to the MongoDB database, and check whether the file node already exists according to the absolute path name of the file node to be deleted. If the file node does not exist, the MongoDB database will be closed. If the file node exists, it will be deleted from fs Delete the file node from the .nodes collection. When the file node cannot be deleted from the fs.nodes collection, the MongoDB database will be closed directly. When the file node is successfully deleted from the fs.nodes collection, a MongoDB GridFS object will be created. Pass The MongoDB GridFS object deletes the file information corresponding to the file node from the MongoDB database.
参考图5，所述目录节点的删除包括以下步骤：With reference to Figure 5, the deletion of the directory node includes the following steps:
将Linux文件系统连接MongoDB数据库，根据所需删除的目录节点的绝对路径名称查找是否已存在该目录节点。当不存在该目录节点，则关闭MongoDB数据库，当存在该目录节点时，则计算该目录节点的父节点绝对路径名称，当该目录节点的父节点绝对路径名称为根目录时，则关闭MongoDB数据库，当目录节点的父节点绝对路径名称不为根目录时，则修改父节点中连接数字段的值，当不能修改父节点中连接数字段的值时，则关闭MongoDB数据库，当成功修改父节点中连接数字段的值时，则针对当前节点的所有子节点，依据子节点类型，递归调用删除文件节点中的Step 1-Step 5或递归调用删除目录节点中的Step 1-Step6。Connect the Linux file system to the MongoDB database, and check whether the directory node already exists according to the absolute path name of the directory node to be deleted. When the directory node does not exist, the MongoDB database is closed. When the directory node exists, the absolute path name of the parent node of the directory node is calculated. When the absolute path name of the parent node of the directory node is the root directory, the MongoDB database is closed. , when the absolute path name of the parent node of the directory node is not the root directory, modify the value of the connection number field in the parent node, and close the MongoDB database when the value of the connection number field in the parent node cannot be modified, and when the parent node is successfully modified When the value of the connection number field is set, for all child nodes of the current node, recursively call Step 1-Step 5 in the delete file node or recursively call Step 1-Step 6 in the delete directory node according to the child node type.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410469287.2ACN104239511B (en) | 2014-09-15 | 2014-09-15 | A MongoDB-oriented user space file system implementation method |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410469287.2ACN104239511B (en) | 2014-09-15 | 2014-09-15 | A MongoDB-oriented user space file system implementation method |
| Publication Number | Publication Date |
|---|---|
| CN104239511Atrue CN104239511A (en) | 2014-12-24 |
| CN104239511B CN104239511B (en) | 2016-03-30 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410469287.2AExpired - Fee RelatedCN104239511B (en) | 2014-09-15 | 2014-09-15 | A MongoDB-oriented user space file system implementation method |
| Country | Link |
|---|---|
| CN (1) | CN104239511B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106027562A (en)* | 2016-07-08 | 2016-10-12 | 中国银联股份有限公司 | Network mapping method and device for files |
| CN106980618A (en)* | 2016-01-15 | 2017-07-25 | 航天信息股份有限公司 | File memory method and system based on MongoDB distributed type assemblies frameworks |
| CN107766355A (en)* | 2016-08-17 | 2018-03-06 | 腾讯科技（深圳）有限公司 | Hierarchical data management method, hierarchical data management system and instantaneous communication system |
| CN107807932A (en)* | 2016-09-08 | 2018-03-16 | 腾讯科技（深圳）有限公司 | A kind of hierarchical data management method and system based on path enumeration |
| CN108268609A (en)* | 2017-12-29 | 2018-07-10 | 新浪网技术(中国)有限公司 | A kind of foundation of file path, access method and device |
| CN110377598A (en)* | 2018-04-11 | 2019-10-25 | 西安邮电大学 | A kind of multi-source heterogeneous date storage method based on intelligence manufacture process |
| CN113065774A (en)* | 2021-04-06 | 2021-07-02 | 中国电建集团成都勘测设计研究院有限公司 | Water conservancy and hydropower engineering investigation and design result management method |
| CN113377721A (en)* | 2021-07-02 | 2021-09-10 | 电信科学技术第五研究所有限公司 | File table design method for storing files in database |
| CN113672562A (en)* | 2020-05-14 | 2021-11-19 | 北京字节跳动网络技术有限公司 | Data deleting method, device, equipment and storage medium |
| WO2023006166A1 (en)* | 2021-07-26 | 2023-02-02 | Марат Рашидович АБЕНОВ | System and method for organizing a service for the sale of goods and services |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216850A (en)* | 2008-01-11 | 2008-07-09 | 清华大学 | A method for dynamic collection of file system access records |
| CN101382953A (en)* | 2008-09-19 | 2009-03-11 | 中兴通讯股份有限公司 | Interface system for accessing file system in user space and file reading and writing method |
| CN102609464A (en)* | 2012-01-16 | 2012-07-25 | 北京亿赞普网络技术有限公司 | Method and device for associative table query of MONGODB shards |
| CN103207919A (en)* | 2013-04-26 | 2013-07-17 | 北京亿赞普网络技术有限公司 | Method and device for quickly inquiring and calculating MangoDB cluster |
| CN103383660A (en)* | 2013-04-01 | 2013-11-06 | 华为终端有限公司 | Method and device for realizing functions of cache partition of Android system as well as terminal equipment |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216850A (en)* | 2008-01-11 | 2008-07-09 | 清华大学 | A method for dynamic collection of file system access records |
| CN101382953A (en)* | 2008-09-19 | 2009-03-11 | 中兴通讯股份有限公司 | Interface system for accessing file system in user space and file reading and writing method |
| CN102609464A (en)* | 2012-01-16 | 2012-07-25 | 北京亿赞普网络技术有限公司 | Method and device for associative table query of MONGODB shards |
| CN103383660A (en)* | 2013-04-01 | 2013-11-06 | 华为终端有限公司 | Method and device for realizing functions of cache partition of Android system as well as terminal equipment |
| CN103207919A (en)* | 2013-04-26 | 2013-07-17 | 北京亿赞普网络技术有限公司 | Method and device for quickly inquiring and calculating MangoDB cluster |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106980618A (en)* | 2016-01-15 | 2017-07-25 | 航天信息股份有限公司 | File memory method and system based on MongoDB distributed type assemblies frameworks |
| CN106027562A (en)* | 2016-07-08 | 2016-10-12 | 中国银联股份有限公司 | Network mapping method and device for files |
| CN107766355A (en)* | 2016-08-17 | 2018-03-06 | 腾讯科技（深圳）有限公司 | Hierarchical data management method, hierarchical data management system and instantaneous communication system |
| CN107766355B (en)* | 2016-08-17 | 2021-07-16 | 腾讯科技（深圳）有限公司 | Hierarchical data management method, hierarchical data management system and instant messaging system |
| CN107807932A (en)* | 2016-09-08 | 2018-03-16 | 腾讯科技（深圳）有限公司 | A kind of hierarchical data management method and system based on path enumeration |
| CN108268609A (en)* | 2017-12-29 | 2018-07-10 | 新浪网技术(中国)有限公司 | A kind of foundation of file path, access method and device |
| CN108268609B (en)* | 2017-12-29 | 2022-01-28 | 星潮闪耀移动网络科技（中国）有限公司 | File path establishing and accessing method and device |
| CN110377598A (en)* | 2018-04-11 | 2019-10-25 | 西安邮电大学 | A kind of multi-source heterogeneous date storage method based on intelligence manufacture process |
| CN113672562A (en)* | 2020-05-14 | 2021-11-19 | 北京字节跳动网络技术有限公司 | Data deleting method, device, equipment and storage medium |
| CN113672562B (en)* | 2020-05-14 | 2024-01-16 | 抖音视界有限公司 | Data deleting method, device, equipment and storage medium |
| CN113065774A (en)* | 2021-04-06 | 2021-07-02 | 中国电建集团成都勘测设计研究院有限公司 | Water conservancy and hydropower engineering investigation and design result management method |
| CN113377721A (en)* | 2021-07-02 | 2021-09-10 | 电信科学技术第五研究所有限公司 | File table design method for storing files in database |
| CN113377721B (en)* | 2021-07-02 | 2023-03-24 | 电信科学技术第五研究所有限公司 | File table design method for storing files in database |
| WO2023006166A1 (en)* | 2021-07-26 | 2023-02-02 | Марат Рашидович АБЕНОВ | System and method for organizing a service for the sale of goods and services |
| Publication number | Publication date |
|---|---|
| CN104239511B (en) | 2016-03-30 |
| Publication | Publication Date | Title |
|---|---|---|
| CN104239511B (en) | A MongoDB-oriented user space file system implementation method | |
| US10740319B2 (en) | Graph-based data models for partitioned data | |
| US10037341B1 (en) | Nesting tree quotas within a filesystem | |
| CN113986873B (en) | Method for processing, storing and sharing data modeling of mass Internet of things | |
| CN103002027B (en) | Data-storage system and the method for tree directory structure is realized based on key-value pair system | |
| CN103229173B (en) | Metadata management method and system | |
| CN106484820B (en) | A renaming method, access method and device | |
| CN111917834A (en) | A data synchronization method, device, storage medium and computer equipment | |
| US8321487B1 (en) | Recovery of directory information | |
| CN103595797B (en) | Caching method for distributed storage system | |
| CN103793534A (en) | Distributed file system and implementation method for balancing storage loads and access loads of metadata | |
| CN103714123A (en) | Methods for deleting duplicated data and controlling reassembly versions of cloud storage segmented objects of enterprise | |
| CN111522791B (en) | Distributed file repeated data deleting system and method | |
| CN103023982A (en) | Low-latency metadata access method of cloud storage client | |
| CN109522271B (en) | Batch insertion and deletion method and device for B + tree nodes | |
| WO2009021465A1 (en) | Method and device for storing document data in document bank system | |
| WO2014110940A1 (en) | A method, apparatus and system for storing, reading the directory index | |
| CN103617199A (en) | Data operating method and data operating system | |
| CN107977396A (en) | A kind of update method of the tables of data of KeyValue databases and table data update apparatus | |
| CN107766355B (en) | Hierarchical data management method, hierarchical data management system and instant messaging system | |
| CN111104377A (en) | File management method, electronic device and computer-readable storage medium | |
| CN104217011A (en) | Method and device for inquiring HBase secondary index table | |
| CN107077511A (en) | Apparatus and method for creating user-defined variable-size labels on records in RDBMS | |
| US11977460B1 (en) | File system namespace reconstruction | |
| CN105843809B (en) | Data processing method and device |
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | Granted publication date:20160330 | |
| CF01 | Termination of patent right due to non-payment of annual fee |