CN109857703B - Pocket space duty ratio statistical method and device based on minium file system - Google Patents

Abstract

The invention relates to a pocket space duty ratio statistical method based on a minium file system, which comprises the steps of connecting a remote file control system; identifying and generating a socket list; extracting the socket cell information; traversing the socket unit and generating a hierarchy list; extracting hierarchy information; traversing the hierarchy and generating a file list; extracting file information; traversing the file and generating configuration information; generating a first-level report; analyzing the first-level report and generating a reference list; and uploading the reference list to a server. A pocket space duty cycle statistics device based on a mini file system, comprising: the system comprises a loading module, a total traversing module, a socket calling module, a socket traversing module, a hierarchy calling module, a hierarchy traversing module, a file calling module, a file traversing module, a preprocessing module, an analyzing module and an uploading module. The invention can carry out effective space occupation ratio statistics on the socket space in the storage space of the minio file system, thereby greatly facilitating the management of the file system by staff and improving the working efficiency.

Description

Pocket space duty ratio statistical method and device based on minium file system

Technical Field

The invention relates to a bucket space duty ratio statistical method and a device thereof based on a minium file system, belonging to the field of server testing.

Background

Under the existing large business scene, the Minio file system is more and more widely used. As an open-source multi-platform file system, minio can provide a plurality of interfaces for uploading, downloading, modifying and the like of files, so as to realize lightweight file operation; the object storage service based on Apache License v2.0 open source protocol can be compatible with an Amazon S3 cloud storage service interface, and is very suitable for storing high-capacity unstructured data, such as pictures, videos, log files, backup data, virtual machine images and the like; there is no explicit limitation on the size of the object file, and it varies from a few kb to a maximum of 5T. Meanwhile, minio is used as a very light service, and can be easily combined with NodeJS, redis or MySQL and the like.

However, when the background compression measurement platform uses a minifile system as a file storage medium, a problem exists in how to effectively acquire space occupation statistical information; if the occupied space of all the socket units is required to be counted, the source system does not provide relevant means. The technical means at the present stage is to enter the root directory of the files in the machine and check the space use condition by inputting the du-sh command.

However, as a file system, data cleaning is necessarily involved; however, if effective data cleaning is to be realized, at least the occupied space and the latest modification time of each file need to be acquired, so that the disk space can be cleaned in a targeted manner, and how to realize the effect is a problem to be solved at present.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a pocket space duty ratio statistical method based on a mini file system, which comprises the following steps:

s1, connecting a remote file control system;

s2, traversing the storage space, identifying a socket unit and generating a socket list;

s3, calling a socket list and extracting socket unit information;

s4, traversing different socket units based on the socket unit information respectively and generating a corresponding level list;

s5, calling different level lists and extracting corresponding level information;

s6, traversing different levels based on the level information and generating a corresponding file list;

s7, calling different file lists and extracting corresponding file information;

s8, traversing different files based on the file information and generating corresponding configuration information;

s9, preprocessing all configuration information and generating a first-level report;

s10, carrying out further data processing on the primary report based on preset information and generating a reference list;

s11, uploading the reference list to a server.

Further, the storage space comprises at least one socket unit.

Further, the bucket unit information at least comprises a bucket name, a bucket address, the amount of occupied storage space and the latest modification time.

Further, the socket unit comprises at least one hierarchy.

Further, the hierarchy information includes at least a hierarchy name, a hierarchy address, and an amount of occupied storage space.

Further, the hierarchy includes at least one file.

Further, the file information includes at least a file name, a file address, and an amount of occupied storage space.

Further, the generation of the primary list specifically refers to sorting of all the socket units, corresponding levels of different socket units and corresponding file information of different levels in the storage space according to the traversal time.

Further, the preset information at least comprises a sequence of storage time, a sequence of occupied storage space and a duty ratio of occupied storage space.

A pocket space duty cycle statistics device based on a mini file system, comprising:

the loading module is used for connecting a remote file control system;

the general traversal module is used for traversing the storage space, identifying the socket units and generating a socket list;

the socket calling module is used for calling the socket list and extracting the socket unit information;

the socket traversing module is used for traversing different socket units respectively and generating a corresponding level list;

the hierarchy calling module is used for calling different hierarchy lists and extracting corresponding hierarchy information;

the hierarchy traversing module is used for traversing different hierarchies respectively and generating a corresponding file list;

the file calling module is used for calling different file lists and extracting corresponding file information;

the file traversing module is used for traversing different files respectively and generating corresponding configuration information;

the preprocessing module is used for preprocessing all the configuration information and generating a first-level report;

the analysis module is used for further data processing of the first-level report and generating a reference list; and

and the uploading module is used for uploading the reference list to the server.

The beneficial effects of the invention are as follows: the efficient space occupation ratio statistics can be carried out on the socket space in the storage space of the mini file system, so that the management of workers on the file system is greatly facilitated, and the working efficiency is improved.

Drawings

FIG. 1 is a general flow chart according to the present invention;

FIG. 2 is a diagram illustrating a device connection according to the present invention;

FIG. 3 is a schematic diagram of embodiment A according to the present invention;

fig. 4 is a schematic diagram of an embodiment B according to the present invention.

Detailed Description

It should be appreciated that embodiments of the invention may be implemented or realized by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The methods may be implemented in a computer program using standard programming techniques, including a non-transitory computer readable storage medium configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner, in accordance with the methods and drawings described in the specific embodiments. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.

Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes (or variations and/or combinations thereof) described herein may be performed under control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications), by hardware, or combinations thereof, collectively executing on one or more processors. The computer program includes a plurality of instructions executable by one or more processors.

Further, the method may be implemented in any type of computing platform operatively connected to a suitable computing platform, including, but not limited to, a personal computer, mini-computer, mainframe, workstation, network or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and so forth. Aspects of the invention may be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and/or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer, which when read by a computer, is operable to configure and operate the computer to perform the processes described herein. Further, the machine readable code, or portions thereof, may be transmitted over a wired or wireless network. When such media includes instructions or programs that, in conjunction with a microprocessor or other data processor, implement the steps described above, the invention described herein includes these and other different types of non-transitory computer-readable storage media. The invention also includes the computer itself when programmed according to the methods and techniques of the present invention.

The computer program can be applied to the input data to perform the functions described herein, thereby converting the input data to generate output data that is stored to the non-volatile memory. The output information may also be applied to one or more output devices such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects produced on a display.

It is noted that, as used in this disclosure, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the description presented herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and/or" as used herein includes any combination of one or more of the listed items.

It is to be understood that the use of any and all examples, or exemplary language (e.g., "such as") provided herein, is intended merely to better illuminate embodiments of the invention and does not pose a limitation on the scope of the invention unless otherwise claimed.

Specific embodiments of the present invention will be further described with reference to the accompanying drawings:

referring to fig. 1, there is shown a general flow chart according to the present invention, comprising the steps of:

s1, connecting a remote file control system;

s2, traversing the storage space, identifying a socket unit and generating a socket list; the storage space at least comprises a socket unit;

s3, calling a socket list and extracting socket unit information; the socket unit information at least comprises a socket name, a socket address, the occupied storage space amount and the latest modification time;

s4, traversing different socket units based on the socket unit information respectively and generating a corresponding level list; the socket unit at least comprises a hierarchy;

s5, calling different level lists and extracting corresponding level information; the hierarchy information at least comprises a hierarchy name, a hierarchy address and the amount of occupied storage space;

s6, traversing different levels based on the level information and generating a corresponding file list;

s7, calling different file lists and extracting corresponding file information; the hierarchy includes at least one file;

s8, traversing different files based on the file information and generating corresponding configuration information; the file information at least comprises a file name, a file address and the amount of occupied storage space;

s9, preprocessing all configuration information and generating a first-level report; the generation of the first-level list specifically refers to sorting of all the socket units, corresponding levels of different socket units and corresponding file information of different levels in the storage space according to the traversal time;

s10, carrying out further data processing on the primary report based on preset information and generating a reference list; the preset information at least comprises the sequence of storage time, the sequence of occupied storage space and the duty ratio of occupied storage space;

s11, uploading the reference list to a server.

Referring to fig. 2, there is shown a device connection diagram according to the present invention, comprising in particular the following parts:

the loading module is connected with the total traversal module to realize interaction and is used for connecting a remote file control system;

the general traversing module is connected with the socket calling module to realize interaction and is used for traversing the storage space, identifying socket units and generating a socket list;

the socket calling module is connected with the socket traversing module to realize interaction and is used for calling a socket list and extracting socket unit information;

the socket traversing module is connected with the hierarchy calling module to realize interaction and is used for respectively traversing different socket units and generating a corresponding hierarchy list;

the hierarchy calling module is connected with the hierarchy traversing module to realize interaction and is used for calling different hierarchy lists and extracting corresponding hierarchy information;

the hierarchy traversing module is connected with the file calling module to realize interaction and is used for respectively traversing different hierarchies and generating a corresponding file list;

the file calling module is connected with the file traversing module to realize interaction and is used for calling different file lists and extracting corresponding file information;

the file traversing module is connected with the preprocessing module to realize interaction and is used for traversing different files respectively and generating corresponding configuration information;

the preprocessing module is connected with the analysis module to realize interaction and is used for preprocessing all configuration information and generating a first-level report;

the analysis module is connected with the uploading module to realize interaction and is used for carrying out further data processing on the primary report and generating a reference list; and

and the uploading module is used for uploading the reference list to the server.

Referring to fig. 3, a schematic diagram of an embodiment a of the present invention is shown, where the embodiment a shows a list of different socket units in a minio file system, and the list may be ordered according to the size of the occupied space or the latest modification time, and the storage status of the storage space may be clearly shown in a list form; in an actual use environment, the program can acquire data in a certain period or times, and is suitable for long-term maintenance of the server.

Referring to fig. 4, a schematic diagram of an embodiment B of the present invention is shown, where an embodiment B shows a table further beautified based on a list in an embodiment a, and in an actual use environment, a worker may send a disk occupation of a socket to an operation and maintenance or other relevant practitioner periodically through this form.

The present invention is not limited to the above embodiments, but is merely preferred embodiments of the present invention, and the present invention should be construed as being limited to the above embodiments as long as the technical effects of the present invention are achieved by the same means. Various modifications and variations are possible in the technical solution and/or in the embodiments within the scope of the invention.

Claims (10)

1. A pocket space duty ratio statistical method based on a minium file system comprises the following steps:

s1, connecting a remote file control system;

s2, traversing the storage space, identifying a socket unit and generating a socket list;

s3, calling a socket list and extracting socket unit information;

s4, traversing different socket units based on the socket unit information respectively and generating a corresponding level list;

s5, calling different level lists and extracting corresponding level information;

s6, traversing different levels based on the level information and generating a corresponding file list;

s7, calling different file lists and extracting corresponding file information;

s8, traversing different files based on the file information and generating corresponding configuration information;

s9, preprocessing all configuration information and generating a first-level report;

s10, carrying out further data processing on the primary report based on preset information and generating a reference list;

s11, uploading the reference list to a server.

2. The method for counting the bucket space duty cycle based on a minium file system according to claim 1, wherein the storage space comprises at least one bucket unit.

3. The method for counting the bucket space occupation ratio based on the minium file system according to claim 1, wherein the bucket unit information at least comprises bucket names, bucket addresses, occupied storage space amounts and latest modification time.

4. The method of claim 1, wherein the bucket units comprise at least one hierarchy.

5. The method of claim 1, wherein the hierarchy information includes at least a hierarchy name, a hierarchy address, and an amount of storage space occupied.

6. The method of claim 1, wherein the hierarchy includes at least one file.

7. The method of claim 1, wherein the file information comprises at least a file name, a file address, and an amount of storage space occupied.

8. The method for counting the bucket space occupation ratio based on the minium file system according to claim 1, wherein the generation of the primary report specifically means ordering all bucket units, corresponding levels of different bucket units and corresponding file information of different levels in the storage space according to traversal time.

9. The method for counting the bucket space duty ratio based on the minium file system according to claim 1, wherein the preset information at least comprises a sequence of storage time, a sequence of occupied storage space amount and a duty ratio of occupied storage space amount.

10. A pocket space duty cycle statistics device based on a mini file system, comprising: the loading module is used for connecting a remote file control system;

the general traversal module is used for traversing the storage space, identifying the socket units and generating a socket list; the socket calling module is used for calling the socket list and extracting the socket unit information;

the socket traversing module is used for traversing different socket units respectively and generating a corresponding level list; the hierarchy calling module is used for calling different hierarchy lists and extracting corresponding hierarchy information;

the hierarchy traversing module is used for traversing different hierarchies respectively and generating a corresponding file list;

the file calling module is used for calling different file lists and extracting corresponding file information;

the file traversing module is used for traversing different files respectively and generating corresponding configuration information;

the preprocessing module is used for preprocessing all the configuration information and generating a first-level report;

the analysis module is used for further data processing of the first-level report and generating a reference list; and

and the uploading module is used for uploading the reference list to the server.

Images