US20130347127A1 - Database management by analyzing usage of database fields - Google Patents

Abstract

Apparatuses, computer readable media, methods, and systems are described for accessing table layout data identifying a plurality of fields included in a table stored in a database, accessing log data identifying a number of times each of the fields of the table has been queried, ranking the fields based on the number of times each of the fields of the table has been queried, determining a first subset of the fields that have not been queried at least a predetermined number of times, and generating an update to the table by removing the first subset of the fields from the table.

Description

BACKGROUND
• A database is an organized collection of data stored in one or more tables each including multiple fields. Due to the size of some databases, queries run against databases may be time and resource consumptive. Moreover, databases often store metadata that contains information describing data stored in the data fields. Metadata may become difficult to manage if there are a large number of data fields. Maintaining data security can also be an issue for databases. In view of such issues, improvements in database management and data security are needed.
BRIEF SUMMARY
• The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosure. The summary is not an extensive overview of the disclosure. It is neither intended to identify key or critical elements of the disclosure nor to delineate the scope of the disclosure. The following summary merely presents some concepts of the disclosure in a simplified form as a prelude to the description below.
• According to aspects of example embodiments, apparatuses, computer readable media, methods, and systems are described for database management by analyzing usage of database fields.
• In some aspects, apparatuses, computer readable media, methods, and systems are described for accessing table layout data identifying a plurality of fields included in a table stored in a database, accessing log data identifying a number of times each of the fields of the table has been queried, and ranking the fields based on the number of times each of the fields of the table has been queried. In some additional aspects, apparatuses, computer readable media, methods, and systems are described for determining a first subset of the fields that have not been queried at least a predetermined number of times, and generating an update to the table by removing the first subset of the fields from the table.
• Aspects of the embodiments may be provided in at least one computer-readable medium and/or memory storing computer-readable instructions that, when executed by at least one processor, cause a computer or other apparatus to perform one or more of the operations described herein.
• These and other aspects of the embodiments are discussed in greater detail throughout this disclosure, including the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
• The present disclosure is illustrated by way of example and not limited in the accompanying figures in which like reference numerals indicate similar elements and in which:
• FIG. 1 shows an illustrative operating environment in which various aspects of the present disclosure may be implemented, in accordance with example embodiments.
• FIG. 2 is an illustrative block diagram of workstations and servers that may be used to implement the processes and functions of certain aspects of the present disclosure, in accordance with example embodiments.
• FIG. 3 illustrates an example process flow diagram for analyzing tables of a database, in accordance with example embodiments.
• FIG. 4 illustrates an example of a database table, in accordance with example embodiments.
• FIG. 5 illustrates an example chart providing usage information based on which departments have been accessing a particular table stored in a database, in accordance with example embodiments.
• FIG. 6 illustrates an example chart providing usage information based on individual users that have been accessing a particular table stored in a database, in accordance with example embodiments.
• FIG. 7 illustrates an example chart providing usage information of multiple tables stored in a database, in accordance with example embodiments.
• FIG. 8 illustrates an example pareto chart in accordance with example embodiments.
• FIG. 9 illustrates an example flow diagram of a method, in accordance with example embodiments.
DETAILED DESCRIPTION
• In the following description of the various embodiments, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration various embodiments in which the disclosure may be practiced. It is to be understood that other embodiments may be utilized and structural and functional modifications may be made without departing from the scope and spirit of the present disclosure.
• FIG. 1 illustrates an example of a suitablecomputing system environment100 that may be used according to one or more illustrative embodiments. Thecomputing system environment100 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality contained in the disclosure. Thecomputing system environment100 should not be interpreted as having any dependency or requirement relating to any one or combination of components shown in the illustrativecomputing system environment100.
• With reference toFIG. 1, thecomputing system environment100 may include acomputing device101 having aprocessor103 for controlling overall operation of thecomputing device101 and its associated components, including random-access memory (RAM)105, read-only memory (ROM)107,communications module109, andmemory115.Computing device101 may include a variety of computer readable media. Computer readable media may be any available media that may be accessed bycomputing device101, may be non-transitory, and may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, object code, data structures, program modules, or other data. Examples of computer readable media may include random access memory (RAM), read only memory (ROM), electronically erasable programmable read only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed bycomputing device101.
• Although not required, various aspects described herein may be embodied as a method, a data processing system, or as a computer-readable medium storing computer-readable instructions. For example, a computer-readable medium storing instructions to cause a processor to perform steps of a method in accordance with aspects of the disclosed embodiments is contemplated. For example, aspects of the method steps disclosed herein may be executed on a processor oncomputing device101. Such a processor may execute computer-readable instructions stored on a computer-readable medium.
• Software may be stored withinmemory115 and/or storage to provide instructions toprocessor103 for enablingcomputing device101 to perform various functions. For example,memory115 may store software used by thecomputing device101, such as anoperating system117,application programs119, and an associateddatabase121. Also, some or all of the computer readable instructions forcomputing device101 may be embodied in hardware or firmware. Although not shown,RAM105 may include one or more are applications representing the application data stored inRAM105 while the computing device is on and corresponding software applications (e.g., software tasks), are running on thecomputing device101.
• Communications module109 may include a microphone, keypad, touch screen, and/or stylus through which a user ofcomputing device101 may provide input, and may also include one or more of a speaker for providing audio output and a video display device for providing textual, audiovisual and/or graphical output.Computing system environment100 may also include optical scanners (not shown). Exemplary usages include scanning and converting paper documents, e.g., correspondence, receipts, and the like, to digital files.
• Computing device101 may operate in a networked environment supporting connections to one or more remote computing devices, such ascomputing devices141,151, and161. Thecomputing devices141,151, and161 may be personal computing devices or servers that include many or all of the elements described above relative to thecomputing device101.Computing device161 may be a mobile device (e.g., smart phone) communicating overwireless carrier channel171.
• The network connections depicted inFIG. 1 may include a local area network (LAN)125 and a wide area network (WAN)129, as well as other networks. When used in a LAN networking environment,computing device101 may be connected to theLAN125 through a network interface or adapter in thecommunications module109. When used in a WAN networking environment,computing device101 may include a modem in thecommunications module109 or other means for establishing communications over theWAN129, such as the Internet131 or other type of computer network. It will be appreciated that the network connections shown are illustrative and other means of establishing a communications link between the computing devices may be used. Various well-known protocols such as TCP/IP, Ethernet, FTP, HTTP and the like may be used, and the system can be operated in a client-server configuration to permit a user to retrieve web pages from a web-based server. Any of various conventional web browsers can be used to display and manipulate data on web pages.
• The disclosure is operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the disclosed embodiments include, but are not limited to, personal computers (PCs), server computers, hand-held or laptop devices, smart phones, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
• Referring toFIG. 2, anillustrative system200 for implementing example embodiments according to the present disclosure is shown. As illustrated,system200 may include one ormore workstation computers201.Workstations201 may be local or remote, and may be connected by one ofcommunications links202 tocomputer network203 that is linked viacommunications link205 toserver204. Insystem200,server204 may be any suitable server, processor, computer, or data processing device, or combination of the same.Server204 may be used to process the instructions received from, and the transactions entered into by, one or more participants.
• Computer network203 may be any suitable computer network including the Internet, an intranet, a wide-area network (WAN), a local-area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), or any combination of any of the same.Communications links202 and205 may be any communications links suitable for communicating betweenworkstations201 andserver204, such as network links, dial-up links, wireless links, hard-wired links, as well as network types developed in the future, and the like.
• FIG. 3 illustrates an example process flow diagram for analyzing tables of a database, in accordance with example embodiments. There are several aspects to consider when managing data stored in a database, including data quality, metadata, data security, and data storage. Data quality may be a measure of completeness of the data, compliance with formats for the data, and accuracy of the data. Metadata may be a description of data elements stored in fields of the database. Data security may relate to limiting access to data stored in a database to select individuals. Data storage may relate to determining where data currently is stored, and where data should be stored. Rather than relying on hunches for managing data stored in a database, the example embodiments provide for database management that may account for data quality, metadata, data security, and data storage.
• Computing device101 may periodically execute one or more query scripts302 to obtain information fromdatabase121 about one or more database tables. For example,computing device101 may issue afirst query script302A to retrievelog data306 and asecond query script302B to obtaintable layout data304 associated with a particular database table. Eachquery script302A-B may identify the particular table stored within thedatabase121. For example, a query script may be generated in accordance with a structured query language. Other query languages may also be used.
• An example database table400 is shown inFIG. 4 and includes columns402-412. Each of columns402-412 may include one or more data fields, and each field may include a data element. For example,column402 includes data fields DF1_1 to DF6_1,column406 includes data fields DF1_2 to DF6_2, andcolumn410 includes data fields DF1_3 to DF6_3.Data field414 has been bolded to call out a single data field. Table400 may also include metadata describing some or all of the data fields. For instance,column404 includes metadata for data fields DF1_1 to DF6_1,column408 includes metadata for data fields DF1_2 to DF6_2, andcolumn412 includes metadata for data fields DF1_3 to DF6_3. Metadata may include information describing a data element within a data field. Metadata may include, for example, information describing one or more of a name of the data field (e.g., account number, customer name, home address, and the like), a format of a data element included in the data field, and a description field providing a textual description of the data field (e.g., “this database field includes an account number of a customer's checking account”), and the like.
• Table layout data304 may identify how table400 is organized. For example, table400 may include a fixed number of columns, and an arbitrary number of rows in each column. The number of rows may vary from column to column based on the number of data fields added to a particular column. Each column may have a differing number of rows compared to the other columns, or two or more columns may have the same number of rows.Table layout data304 may generally identify the content of each column. For example, a first column may include customer name data, a second column may include account number data, a third column may include balance data, and the like.
• Logdata306 may provide a time history identifying each time each data field was accessed, a usage history indicating how many times each data field was accessed, and an access history identifying each user and/or organization that accessed any of the data fields.
• Time history, for example, may identify the time at which each data field of table400 was accessed, including the time each data field of table400 was last accessed. For example, time history may indicate thatdata field414 was queried on Jun. 2, 2009 at 10:30 AM, on Apr. 21, 2010 at 11:45 AM, and on Oct. 12, 2011 at 9:42 AM.
• Usage history, for example, may include a count identifying how many times each data field was accessed. For example, usage history may indicate thatdata field414 has been accessed 17 times.
• Access history, for example, may identify each user who accessed a particular data element, as well as any organizational information associated with each user. Organizational information may identify a title of a user (e.g., vice president, manager, accountant, and the like) as well as within what part of the organization the user is associated (e.g., accounting department, sales department, and the like). For example, access history may indicate thatdata field414 was accessed by user52 of the sales department, user98 of the account department, and user217 of the marketing department.
• Computing device101 may execute astatistical analysis script308 to processtable layout data304 and logdata306 for managing table400.Statistical analysis script308 may provide information for monitoring data security, determining for which data field(s) to provide metadata, determining usage of the data fields, determining for which fields to authorize performance of quality review, and determining where to store data elements included in the data fields. Example output of thestatistical analysis script308 is shown inFIGS. 5-8.
• FIG. 5 illustrates an example chart providing usage information based on which departments have been accessing a particular table stored in a database, in accordance with example embodiments.Column502 may identify at least some of the departments that have been accessing table400.Column504 may include a query count identifying the number of times that each department has queried table400. For instance,Department 1 made 2345 queries of table400,Department 2 made 2316 queries of table400, and so forth.Column502 may be ranked so that the department having the largest query count is listed first, the department having the next largest query count is listed next, and so forth.Column506 may identify a cumulative count that is a sum of query count values from a corresponding row and those rows above. For example, in therow including department 3, the cumulative count is 6231 (i.e., 2345+2316+1570).
• FIG. 6 illustrates an example chart providing usage information based on individual users that have been accessing a particular table stored in a database, in accordance with example embodiments.Column602 may identify at least some of the users that have been accessing table400.Column604 may include a query count identifying the number of times that each user has queried table400. For instance,User 1 made 921 queries of table400,User 2 made 863 queries of table400, and so forth.Column602 may be ranked so that the user having the largest query count is listed first, the user having the next largest query count is listed next, and so forth.Column606 may identify a cumulative count that is a sum of query count values from a corresponding row and those rows above. For example, in therow including User 3, the cumulative count is 2618 (i.e., 921+863+834).Column608 may identify the percentage of a total number of queries of table400 that were issued by each user. For example, 7.5% of the total queries of table400 were issued byUser 1, 7.03% of the total queries of table400 were issued byUser 2, and so forth.Column610 may identify a cumulative percentage that is a sum of percentage values from a corresponding row and those rows above. For example, in therow including user 3, the cumulative percentage is 21.32 (i.e., 7.5+7.03+6.79).
• FIG. 7 illustrates an example chart providing usage information of multiple tables stored indatabase121 in accordance with example embodiments.Column702 may identify at least some of the tables stored indatabase121.Column704 may include a query count identifying the number of times that each table has been queried. For instance, Table 1 may have been queried 5570 times, Table 2 may have been queried 1500 times, and so forth.Column702 may be ranked so that the table having the largest query count is listed first, the table having the next largest query count is listed next, and so forth.Column706 may identify a cumulative count that is a sum of query count values from a corresponding row and those rows above. For example, in the row including Table 3, the cumulative count is 7822 (i.e., 5570+1500+752).Column708 may identify the percentage of a total number of queries were of each table. For example, 45.35% of the total queries were of Table 1, 12.21% of the total queries were of Table 2, and so forth.Column710 may identify a cumulative percentage that is a sum of percentage values from a corresponding row and those rows above. For example, in therow including user 3, the cumulative percentage is 63.69 (i.e., 45.35+12.21+6.12).
• FIG. 8 illustrates anexample pareto chart800 in accordance with example embodiments.Computing device101 may generatepareto chart800 to illustrate a relationship between percentage and cumulative percentage for a particular variable (e.g., queries by multiple users of a particular table, queries of each table, and the like). For example, the relationship may be between a percentage of total queries of table400 and cumulative percentage, as described above inFIG. 6.FIG. 8 may be used to describe other relationships between percentage and cumulative percentage, such as, for example, the relationship described inFIG. 7. InFIG. 8, Y-axis802 lists values of percentages and Y-axis804 lists values of cumulative percentages.X-axis806 lists the fields in descending percentage order from left to right based on percentage. Bars associated with each of the fields indicate a percentage associated with a corresponding data field, and descend from left to right.Bar810, for example, is associated with FIELD10 and a percentage value of approximately 4.9%.Line808 indicates the cumulative percentage associated with the data fields, and increases from right to left until approaching a cumulative percentage of 100%. As shown inFIG. 8, bar810 associated with FIELD10 on theX-axis806 is associated with a cumulative percentage of 70.22% on Y-axis804.
• Computing device101 may managedatabase121 based on information derived from executingstatistical analysis script308, such as that shown inFIGS. 5-8.Computing device101 may, for example, make determinations relating to data security, creation of metadata, determine which data fields to submit for quality review, and whether to include certain data fields in a particular data table.
• For making determinations relating to data security,computing device101 may determine which users have been accessing a particular table and whether they are authorized to do so. For example,computing device101 may process the access history of thelog data306 indicating each user that accessed any of the data fields of table400. Each user may be associated with a unique user identifier that is included with any query launched by that user.Computing device101 may compare the unique user identifier to a list of users authorized to access table400. If a particular user identifier is not on the authorized list,computing device101 may update a security control to prevent the user associated with that user identifier from launching queries against table400 in the future. Additionally or alternatively,computing device101 may issue a security violation message to inform appropriate personnel that the user improperly accessed table400.
• In another example, the access history of thelog data306 may indicate which organizations have accessed any of the data elements in table400. Each organization (e.g., department) may be associated with a unique organization identifier that is included with any query launched by a user.Computing device101 may compare the unique organization identifier to a list of organizations authorized to access table400. If a particular organization identifier is not on the authorized list,computing device101 may update a security control to prevent some or all users associated with that organization identifier from launching queries against table400 in the future. Additionally or alternatively,computing device101 may issue a security violation message to inform appropriate personnel that a particular organization improperly accessed table400.
• In other aspects,computing device101 may make determinations relating to creation of metadata. In some instances, creation of metadata may involve manual entry of data. This may be labor intensive and time consuming.Computing device101 may process the usage history for table400 to determine for which data fields to create metadata, and for which data fields not to create metadata.
• In an example,computing device101 may process the usage history to identify a subset of the data fields of table400 that have been queried at least a predetermined number of times (e.g., 35 times), and may authorize creation of metadata for the subset, but not the remaining data fields. In another example,computing device101 may process the usage history to identify a subset of the data fields of table400 that have been queried at least a predetermined number of times and have been queried by at least a predetermined number of unique users, and may authorize creation of metadata for the subset, but not the remaining data fields. In yet another example,computing device101 may process the usage history to identify a subset of the data fields of table400 where a cumulative percentage that those fields have been accessed meets or exceeds a percentage threshold (e.g., 80% cumulative percentage), and may authorize creation of metadata for the subset, but not the remaining data fields. A time constraint may also be applied to any of the above examples. For example, the time constraint may specify that the queries must have occurred within a certain amount of time (e.g., last 6 months) to be considered.
• Metadata of a table may also be periodically updated. When a metadata update is desired,computing device101 may use thequery script302B to obtain a current version of thelog data306 that includes an updated usage history.Computing device101 may analyze the updated usage history to determine a new subset of data fields for which to authorize the creation of metadata for the new subset using any of the methods described above. As compared to an earlier identified subset of data fields, the new subset may be unique or may overlap with data fields of the earlier identified subset.
• When a new table is added todatabase121,computing device101 may control for which data fields to create metadata. In some aspects,computing device101 may attempt to identify a table that is similar to the new table for determining which data fields to create metadata. For example,computing device101 may processtable layout data304 of the new table to identify the structure of the new table, and may search throughdatabase121 attempting to identify any other table stored indatabase121 that includes a similar structure and/or includes at least one similar column. If a similar table is identified,computing device101 may determine a subset of the fields of the similar table for which metadata was created.Computing device101 may then identify the same or a similar subset of data fields in the new table.Computing device101 may authorize creation of metadata for that subset, but not the remaining data fields, of the new table. If a similar column is identified, but not a similar table,computing device101 may determine a subset of the fields in the similar column for which metadata was created, and identify a similar field subset in a corresponding column of the new table.Computing device101 may thus assist in intelligent selection of data fields for which to create metadata, and may reduce creation of metadata for data fields that are rarely or infrequently accessed.
• In some aspects,computing device101 may make determinations relating to determining which data fields to submit for quality review. In some instances, data fields may be manually reviewed to determine completeness of data elements included in the data fields, compliance with formats for the data elements, and accuracy of the data elements. Performing such a review may be labor intensive and time consuming.Computing device101 may process the usage history for table400 to determine for which data fields to submit for quality review, and for which data fields not to review for quality.
• In an example,computing device101 may process the usage history to identify a subset of the data fields of table400 that have been queried at least a predetermined number of times (e.g., 35 times), and may authorize performance of quality review for the subset, but not the remaining data fields. In another example,computing device101 may process the usage history to identify a subset of the data fields of table400 that have been queried at least a predetermined number of times and have been queried by at least a predetermined number of unique users, and may authorize quality review for the subset, but not the remaining data fields. In yet another example,computing device101 may process the usage history to identify a subset of the data fields of table400 where a cumulative percentage that those fields have been accessed meets or exceeds a percentage threshold (e.g., 80% cumulative percentage), and may authorize quality review for the subset, but not the remaining data fields. A time constraint may also be applied to any of the above examples. For example, the time constraint may specify that the queries must have occurred within a certain amount of time (e.g., last 6 months) to be considered.
• Data fields may be periodically reviewed for quality. When a data quality review of table400 is desired,computing device101 may use thequery script302B to obtain a current version of thelog data306 that includes an updated usage history.Computing device101 may analyze the updated usage history to determine a new subset of data fields for which to authorize quality review for the new subset using any of the methods described above. As compared to an earlier identified subset of data fields, the new subset may be unique or may overlap with data fields of the earlier identified subset.
• When a new table is added todatabase121,computing device101 may control for which data fields to authorize quality review. In some aspects,computing device101 may attempt to identify a table that is similar to the new table for determining which data fields to authorize quality review. For example,computing device101 may processtable layout data304 of the new table to identify the structure of the new table, and may search throughdatabase121 attempting to identify any other table stored indatabase121 that includes a similar structure and/or includes at least one similar column. If a similar table is identified,computing device101 may determine a subset of the fields of the similar table for which quality review was authorized.Computing device101 may then identify the same or a similar subset of data fields in the new table.Computing device101 may authorize quality review for that subset, but not the remaining data fields, of the new table. If a similar column is identified, but not a similar table,computing device101 may determine a subset of the fields in the similar column for quality review was authorized, and identify a similar field subset in a corresponding column of the new table.Computing device101 may thus assist in intelligent selection of data fields for which to authorize quality review, and may avoid performing quality review of data fields that are rarely accessed.
• In further aspects,computing device101 may make determinations about which data fields to include in table400, and which to remove, to enhance performance ofdatabase121. In an example,computing device101 may process the time history oflog data306 to identify a subset of the data fields that have not been queried at least for a predetermined amount of time.Computing device101 may exclude that subset from table400, or may create a new table excluding that subset. In some examples,computing device101 may transfer data of the subset to a long term storage device, and may delete the subset from table400. The predetermined time period may be adjusted based on cumulative percentage. For example,computing device101 may determine a point in time within which a certain cumulative percentage (e.g., 80%) of the data fields have been accessed at least once.Computing device101 may then identify a subset of data fields that have not been accessed during the time interval between a current time and the determined point in time, and may remove that subset from the table400. For example,computing device101 may determine that 80% of data fields in table400 have been accessed at least once within the past 272 days.Computing device101 may remove data fields from table400 that have not been accessed within the past 272 days. Thus,computing device101 may remove infrequently accessed data fields from table400.Computing device101 may cause the removed subset of data fields to be stored in another storage device, or alternatively may create a new table excluding the subset.
• FIG. 9 illustrates an example flow diagram of a method, in accordance with example embodiments. The method may be implemented by a single apparatus such as, for example, a computer, server, or other computational device. In other aspects, the method may be performed by multiple devices (e.g., multiple computers, multiple processors, and the like). One or more non-transitory computer readable medium may also store computer readable instructions that, when executed by at least one processor, may cause at least one apparatus to perform the method. The order of the blocks shown inFIG. 9 is an example. The blocks may be arranged in other orders, each function described in each block may be performed one or more times, some blocks may be omitted, and/or additional blocks may be added. The method may begin atblock902.
• Inblock902, the method may include accessing table layout data identifying a plurality of fields included in a database table stored in a database. In an example,computing device101 may issue aquery script302B to retrievetable layout data304 indicating a structure of data fields included in table400.
• Inblock904, the method may include accessing log data identifying a number of times each of the fields of the table has been queried. In an example,computing device101 may issue aquery script302A to retrievelog data306 indicating a number of times that each of the data fields in table400 has been queried.
• Inblock906, the method may include ranking the fields based on the number of times each of the fields of the table has been queried. In an example,computing device101 may rank the data fields of table400 from a greatest number of queries to a least number of queries based on the number of times each of the fields of the table has been queried. Ranking may be used to determine which data fields are frequently accessed as well as data fields that are infrequently accessed. In an example,computing device101 may identify data fields that are frequently accessed (e.g., more than a predetermined number of times) and for which metadata has not been generated, and to initiate generation of metadata for those data fields. In another example,computing device101 may identify data fields that are frequently accessed (e.g., more than a predetermined number of times) and to authorize for those data fields to be reviewed for quality.
• Inblock908, the method may include determining a first subset of the fields that have not been queried at least a predetermined number of times. In an example,computing device101 may determine a first subset of the data fields of table400 that have not been queried at least a predetermined number of times (e.g., 35 times). A time constraint to the predetermined number of queries may be applied. For example,computing device101 may determine a first subset of the data fields of table400 that have not been queried at least a predetermined number of times (e.g., 35 times) within the last 6 months.
• Inblock910, the method may include generating an update to the table based on the subset. In an example,computing device101 may remove the first subset of the data fields of table400 to update table400. In some examples,computing device101 may store the first subset in a long term storage device or medium prior to removing (e.g., deleting) the first subset from table400. The method may end, may return to any of the preceding steps, or may repeat one or more times.
• Aspects of the embodiments have been described in terms of illustrative embodiments thereof. Numerous other embodiments, modifications and variations within the scope and spirit of the appended claims will occur to persons of ordinary skill in the art from a review of this disclosure. For example, one of ordinary skill in the art will appreciate that the steps illustrated in the illustrative figures may be performed in other than the recited order, and that one or more steps illustrated may be optional in accordance with aspects of the embodiments.

Claims (20)

1. An apparatus comprising:

at least one processor; and

at least one memory storing computer readable instructions that, when executed by the at least one processor, cause the apparatus at least to:

access table layout data identifying a plurality of fields included in a table stored in a database;

access log data identifying a number of times each of the fields of the table has been queried;

rank the fields based on the number of times each of the fields of the table has been queried;

determine a first subset of the fields that have not been queried at least a predetermined number of times; and

generate an update to the table based on the first subset.

2. The apparatus ofclaim 1, wherein the generating of the update comprising removing the first subset of the fields from the table.

3. The apparatus ofclaim 1, wherein the computer readable instructions, when executed by the at least one processor, cause the apparatus to process the log data to determine a user identifier associated with a user that has initiated at least one query to access the table.

4. The apparatus ofclaim 3, wherein the computer readable instructions, when executed by the at least one processor, cause the apparatus to determine whether the user associated with the user identifier is authorized to access the table.

5. The apparatus ofclaim 4, wherein the computer readable instructions, when executed by the at least one processor, cause the apparatus to update a security control in response to determining that the user does not have authorization to access the table.

6. The apparatus ofclaim 1, wherein the computer readable instructions, when executed by the at least one processor, cause the apparatus to initiate storage of the first subset of the fields in a storage device different from the database.

7. The apparatus ofclaim 1, wherein the computer readable instructions, when executed by the at least one processor, cause the apparatus to determine, based on the ranking, a second subset of the fields that have been queried at least a predetermined number of times.

8. The apparatus ofclaim 7, wherein the computer readable instructions, when executed by the at least one processor, cause the apparatus to determine a particular data field from the second subset of the fields that is not associated with metadata.

9. The apparatus ofclaim 8, wherein the computer readable instructions, when executed by the at least one processor, cause the apparatus to initiate generation of metadata for the particular data field.

10. The apparatus ofclaim 1, wherein the computer readable instructions, when executed by the at least one processor, cause the apparatus to:

determine a time period within which at least a predetermined percentage of the fields have been queried at least once;

determine a second subset of the fields that have not been queried within the time period; and

exclude the second subset from the table.

11. A computer-implemented method comprising:

accessing log data identifying a number of times each of a plurality of fields of a table included in a database has been queried;

ranking, by at least one processor, the fields based on the number of times each of the fields of the table has been queried;

determining, by the at least one processor, a first subset of the fields that have not been queried at least a predetermined number of times and have not been queried by at least a predetermined number of users; and

generating, by the at least one processor, an update to the table by removing the first subset of the fields from the table.

12. The method ofclaim 11, further comprising:

processing the log data to determine a user identifier associated with a user that has initiated at least one query to access the table;

determining whether the user associated with the user identifier is authorized to access the table; and

updating a security control in response to determining that the user does not have authorization to access the table.

13. The method ofclaim 11, further comprising initiating storage of the first subset of the fields in a storage device different from the database.

14. The method ofclaim 11, further comprising:

determining, based on the ranking, a second subset of the fields that have been queried at least a predetermined number of times; and

initiating generation of metadata for a particular data field from the second subset in response to determining that the particular data field is not associated with metadata.

15. The method ofclaim 11, further comprising

determining a time period within which at least a predetermined percentage of the fields have been queried at least once;

determining a second subset of the fields that have not been queried within the time period; and

excluding the second subset from the table.

16. A non-transitory computer readable medium storing computer readable instructions that, when executed, cause an apparatus at least to:

access log data identifying a number of times each of a plurality of fields included in a table stored in a database has been queried;

rank the fields based on the number of times each of the fields of the table has been queried;

determine a first subset of the fields that have not been queried at least a predetermined number of times; and

generate an update to the table based on the first subset.

17. The computer readable medium ofclaim 16, wherein the computer readable instructions, when executed, cause the apparatus to:

process the log data to determine a user identifier associated with a user that has initiated at least one query to access the table;

determine whether the user associated with the user identifier is authorized to access the table; and

update a security control in response to determining that the user does not have authorization to access the table.

18. The computer readable medium ofclaim 16, wherein the computer readable instructions, when executed, cause the apparatus to initiate storage of the first subset of the fields in a storage device different from the database.

19. The computer readable medium ofclaim 16, wherein the computer readable instructions, when executed, cause the apparatus to:

determine, based on the ranking, a second subset of the fields that have been queried at least a predetermined number of times; and

initiate generation of metadata for a particular data field from the second subset in response to determining that the particular data field is not associated with metadata.

20. The computer readable medium ofclaim 16, wherein the computer readable instructions, when executed, cause the apparatus to:

determine a time period within which at least a predetermined percentage of the fields have been queried at least once;

determine a second subset of the fields that have not been queried within the time period; and

exclude the second subset from the table.

Images