US10613938B2 - Data virtualization using copy data tokens - Google Patents

Abstract

Computerized systems and methods are provided for data virtualization using copy data tokens. A data token is stored that defines attributes associated with copy data, including source data, transformation data, and access data for the copy data. The access data is indicative of a set of access settings for the copy data that define how an instance of the copy data is to be created for the user, and a set of access permissions for the copy data that define an access level for the user for the copy data that defines how much of the copy data the user can access. The data token is transmitted to a remote computer storing the copy data based on the source data in the data token. A copy of the copy data that was generated based on the preparation information and the access data is received.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related to U.S. patent application Ser. No. 14/789,360, entitled “INTEGRATING COPY DATA TOKENS WITH SOURCE CODE REPOSITORIES,” filed on the even-date herewith, the entire content of which is herein incorporated by reference.

FIELD

The subject matter disclosed in this application generally relates to data virtualization using copy data tokens.

BACKGROUND

There are often many departments, groups and applications in any company that would like to get access to the data being created in the Production environment. These include Development, Test, Analytics, Compliance, Marketing, among others. Production does not allow external groups to directly access the Production data fearing that they will effect performance and/or integrity of the data. Because of this, the only way to get access is to get a copy.

There are three main methods to getting a copy of the Production data and they have their downsides. The single largest obstacle to getting a copy is finding a time that it can be created. The action of copying the data will create a performance drop and would need to be scheduled and managed as to when it could occur. Assuming the right schedule could be found, a simple copy of the data could be created. If this is continually changing data, then the simple copy method cannot be used. The second approach would be to create a snapshot on the storage array. This is attractive because it is created quickly and can easily be destroyed when no longer needed. The downside to this is that access to the snapshot will share the same storage array resources with the Production data. Although this solves the problem of data integrity because it is a separate copy, it does not solve the problem of performance impact. The final method used is to get the copy from a backup. Companies will protect the production data by doing a daily backup of it. This creates an independent copy of the data. Restoring the data provides a completely independent copy that does not affect the integrity or the performance of the Production data. The downside with this approach is the amount of time it takes to restore the backup. It could take hours to days to weeks to get a copy of a complex and large data set.

A new solution in the market is Copy Data Virtualization. This captures one full copy and then captures incremental change data according to a schedule. Using storage virtualization techniques, it can provide independent copies in seconds to minutes, regardless of complexity and size. This approach meets the requirements or not effecting integrity or performance and also solves the problem of the time it takes to restore from a backup.

Once a copy of the data is available, it may need further processing to protect sensitive data contained within. For example, a database might contain credit card numbers. The data is protected while in the Production environment but if a copy of the database was provided to Development and Test environment, it will have lost many of the protections that exist in the Production environment. According to the requirements of who will be using it, what the need is and what environment it will exist in, a number of transformations to the data might need to take place. This could include subsetting, masking and data quality checks.

Now that there is a copy and it has been transformed, it now needs to be made accessible to the users outside of the Production environment. This process needs to be repeatable, scalable and manageable. The concept of a copy data token is used to create a self-describing entity that can be kept in a library of data sources and accessed in a controlled manner from within and outside of the Production environment.

SUMMARY

Copy data tokens are the center of a self-describing infrastructure that enables data reuse and sharing in a controlled, managed environment. The user of the copy data token gains the benefits of access to data without having to either be knowledgeable about storage management or need to wait for an expert to help. The providers of the data do not have to provide such a high level of management because the system can be managed in a self-service manner. The providers of the data are able to retain the performance, security and integrity of their data while sharing it with a larger audience.

The disclosed subject matter includes a computerized method for using a data token to automatically manage access to copy data associated with the data token, the data token defining attributes for the copy data associated with the data token to facilitate the automatic management of the copy data. The computerized method includes storing, by a computing device with a processor and memory, a data token that defines attributes associated with copy data. The attributes include source data for the copy data indicative of a data source for the copy data, transformation data for the copy data indicative of a set of modifications for the copy data when creating an instance of the copy data, and access data for the copy data. The access data is indicative of a set of access settings for the copy data that define how an instance of the copy data is to be created for the user, and a set of access permissions for the copy data that define an access level for the user for the copy data that defines how much of the copy data the user can access. The computerized method includes transmitting, by the computing device, the data token to a remote computer storing the copy data based on the source data in the data token. The computerized method includes receiving, by the computing device, a copy of the copy data that was generated based on the preparation information and the access data, thereby using the data token to automatically manage access to copy data associated with the data token for the user.

The disclosed subject matter includes a computerized apparatus for using a data token to automatically manage access to copy data associated with the data token, the data token defining attributes for the copy data associated with the data token to facilitate the automatic management of the copy data. The apparatus includes a processor configured to run a module stored in memory that is configured to cause the processor to store a data token that defines attributes associated with copy data, the attributes including source data for the copy data indicative of a data source for the copy data, transformation data for the copy data indicative of a set of modifications for the copy data when creating an instance of the copy data, and access data for the copy data. The access data is indicative of a set of access settings for the copy data that define how an instance of the copy data is to be created for the user, and a set of access permissions for the copy data that define an access level for the user for the copy data that defines how much of the copy data the user can access. The module stored in memory is further configured to cause the processor to transmit the data token to a remote computer storing the copy data based on the source data in the data token. The module stored in memory is further configured to cause the processor to receive a copy of the copy data that was generated based on the preparation information and the access data, thereby using the data token to automatically manage access to copy data associated with the data token for the user.

The disclosed subject matter includes a non-transitory computer readable medium having executable instructions operable to cause an apparatus to store a data token that defines attributes associated with copy data. The attributes include source data for the copy data indicative of a data source for the copy data, transformation data for the copy data indicative of a set of modifications for the copy data when creating an instance of the copy data, and access data for the copy data. The access data is indicative of a set of access settings for the copy data that define how an instance of the copy data is to be created for the user, and a set of access permissions for the copy data that define an access level for the user for the copy data that defines how much of the copy data the user can access. The non-transitory computer readable medium has executable instructions operable to cause the apparatus to transmit the data token to a remote computer storing the copy data based on the source data in the data token. The non-transitory computer readable medium has executable instructions operable to cause the apparatus to receive a copy of the copy data that was generated based on the preparation information and the access data, thereby using the data token to automatically manage access to copy data associated with the data token for the user.

Before explaining example embodiments consistent with the present disclosure in detail, it is to be understood that the disclosure is not limited in its application to the details of constructions and to the arrangements set forth in the following description or illustrated in the drawings. The disclosure is capable of embodiments in addition to those described and is capable of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein, as well as in the abstract, are for the purpose of description and should not be regarded as limiting.

These and other capabilities of embodiments of the disclosed subject matter will be more fully understood after a review of the following figures, detailed description, and claims. It is to be understood that both the foregoing general description and the following detailed description are explanatory only and are not restrictive of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Various objects, features, and advantages of the disclosed subject matter can be more fully appreciated with reference to the following detailed description of the disclosed subject matter when considered in connection with the following drawings.

FIG. 1 illustrates an exemplary creation process for a copy data token, according to some embodiments;

FIG. 2 illustrates an exemplary computerized method to create a copy data token, according to some embodiments;

FIGS. 3A-3E illustrate an exemplary copy data token structure, including, attribute classes, and attribute names, according to some embodiments;

FIG. 4 illustrates exemplary attributes for a copy data token, according to some embodiments;

FIG. 5 illustrates an exemplary system for a copy data token, according to some embodiments;

FIG. 6 illustrates an exemplary computerized copy data token exchange, according to some embodiments;

FIG. 7 illustrates an exemplary set of operations for a copy data token that can be supported by a command line interface, according to some embodiments;

FIG. 8 illustrates an exemplary computerized method for security processing for copy data tokens, according to some embodiments;

FIG. 9 illustrates an exemplary computerized method for security logic for a copy data token, according to some embodiments;

FIG. 10 illustrates an exemplary computerized method for creating a copy data token, according to some embodiments;

FIG. 11 illustrates an exemplary system diagram for using copy data tokens in development and test environments, according to some embodiments;

FIG. 12 illustrates an exemplary application packages for copy data tokens, according to some embodiments;

FIG. 13 is an exemplary computerized method for creating a copy data token for analytics, according to some embodiments;

FIG. 14 is an exemplary system diagram for using copy data tokens for analytics, according to some embodiments;

FIG. 15 is an exemplary computerized method for creating a copy data token for research data, according to some embodiments;

FIG. 16 is an exemplary computerized method for using copy data tokens for research data, according to some embodiments;

FIG. 17 is an exemplary computerized method of problem re-creation for a copy data token, according to some embodiments;

FIG. 18 is an exemplary system diagram for problem re-creation for a copy data token, according to some embodiments; and

FIG. 19 is an exemplary system diagram for using copy data tokens for cloud archiving, according to some embodiments; and

FIG. 20 is an exemplary diagram illustrating copy data token data relationships for copy data tokens, according to some embodiments.

DESCRIPTION

In the following description, numerous specific details are set forth regarding the systems and methods of the disclosed subject matter and the environment in which such systems and methods may operate, in order to provide a thorough understanding of the disclosed subject matter. It will be apparent to one skilled in the art, however, that the disclosed subject matter may be practiced without such specific details, and that certain features, which are well known in the art, are not described in detail in order to avoid complication of the disclosed subject matter. In addition, it will be understood that the embodiments described below are only examples, and that it is contemplated that there are other systems and methods that are within the scope of the disclosed subject matter.

In some exemplary embodiments, a data structure (e.g., referred to as a copy data token) is provided to automatically manage access to copy data associated with the data token. The data token defines attributes for the copy data associated with the data token to facilitate the automatic management of the associated copy data. Each data token can be customized for an associated user or application. Therefore the copy data token can keep track of copy data so that copy data can be automatically managed for each user that needs to access the copy data while still providing customization for the particular user.

In some embodiments, the copy data token includes information that describes what it is, its configuration, its contents, and/or the like. It can be passed to a system to gain access to a data set. By adding to this concept a set of storage functions like snapshot and storage virtualization, a system can be provided that gives users access to a library or collection of Copy Data Tokens which in turn provide access to the data. Using storage virtualization, a single real copy of data can be used to provide multiple copies to multiple users with the same token.

In some embodiments, the copy data token includes sufficient data to provide information necessary to know where the data associated with the token came from, how it was prepared, how to get the data, whether a user could access the data, and/or the like. This template could be used, for example, for both gaining access to data along with running automation to create the copy to be used for reuse.

FIG. 20 illustrates an exemplary diagram of relations of data used in a copy data token system, according to some embodiments. The diagram inFIG. 20 showsproduction data2001,copy data2002 that was created on Jan. 1, 2015 at 10:00 am, copy data2003 that was created on Jan. 1, 2015 at 11:00 am, a transformation process2004 (which is described in further detail herein), a primary copy of data2005 carrying forward the creation time of thecopy data2002, a working copy of data2006 carrying forward the creation time of thecopy data2002, a working copy of data2007 carrying forward the creation time of thecopy data2002, and a working copy of data2008 carrying forward the creation time of thecopy data2002.Production data2001 is any information that is created and stored within the production environment of a data center. It is typically related to ongoing operations of the company or organization.Copy data2002 and2003 are copies of theproduction data2001 that have been copied at a given time.Transformation2004 is a set of operations such as data masking which are used to transform the input data to a new copy on the output side. Primary copy of data2005 is the new copy of data as a result of executing thetransformation2004 operation. Working copy of data2006,2007 and2008 are copies being created by the copy data virtualization system as shown inFIG. 1 as copydata virtualization system104.

A specific version of production data can be captured fromProduction data2001 using a copy data virtualization system and is captured as copy data at Jan. 1, 2015 at 10:00am2002 and at Jan. 1, 2015 11:00 am2003. Thecopy data2002 and2003 are in the same native format as theproduction data2001 and are copies existing outside of the production environment. These copies can then be reused as needed.Copy data2002 is put through atransformation2004 process that may include operations such as data masking with the result being primary copy of data2005. The goal of the transformation process is, for example, to remove data, such as credit card numbers, that cannot exist outside of the production environment. The primary copy of data created at Jan. 1, 2015 10:20 am2005 is an copy of thecopy data2002 with the difference being that credit card numbers have been changed from real numbers to fake numbers. As different users are using the copy data token the system it is automatically creating, for each user, a single working copy of data2006,2007 and2008. These copies are created so the primary copy of data2005 is not modified and can continue to be reused by many people. A user could delete a work copy, for example2006, and then recreate it from the primary copy2005. This allows users to try operations that would modify the working copy2006 and then decide to get rid of the changes by starting over with a fresh copy from primary copy of data2005.

In some embodiments, the techniques use a command line tool or API that provides a set of operations based on the contents of a token. The command line can be added, for example, to plug-ins, triggers or hooks (e.g., names for methods of integration into applications) for integration into applications. This is described in further detail herein.

FIG. 7 illustrates an exemplary set of operations for a copy data token that can be supported by a command line interface, according to some embodiments. The commands shown inFIG. 7 include anedit command750, a status command751, abranch command752, acheckout command753, acheckin command754, a delete command755, and a createcommand756. Anedit command750 is used to modify or create token attributes, e.g., as shown inFIG. 4, copy data token attributes417. A status command751 is used to provide a status of the data in the copy data token system or software, e.g., as show inFIG. 5 as application or command line tool501. Thebranch command752 will cause the copy data virtualization system to make a new working copy of data (e.g.,FIG. 20, working copy of data2006,2007,2008) from the Primary copy of data (e.g.,FIG. 20, primary copy of data2005). Thecheckout command753 will make the working copy of data2006 accessible or not accessible. The checkingcommand754 will make a working copy of data2006 to be promoted to a primary copy of data2005 so it can be preserved and shared. A delete command755 is used to remove a working copy of data2006. A createcommand756 is used to create a new primary copy of data2005.

Edit command750 can edit a field in the copy data token, which takes as input an attribute-class, attribute-name and an attribute-value. An attribute-class is show, for example inFIG. 3 asattribute classes314. An attribute-name is shown, for example, inFIG. 3 asattribute name350 andFIG. 4 asattribute name415. An attribute-value is shown inFIG. 4 at416. It is the value stored associated with anattribute name415 for anattribute417 as part of acopy data token310. The -n option is used to specify a name of a data set, as shown inFIG. 3 asdata set312, in the copy data token as shown inFIG. 3 as copy data token310. This would be used if there is multiple data sets inside a single token, e.g., as shown inFIG. 1 as copy data token102. If the name does not exist, the command will prompt the user if the name data set should be created. If the -f flag is present, it will create the named data set. The -f flag is used to force it to create a new token, e.g., as shown inFIG. 1 as copy data token102. Otherwise, it will prompt if the token does not exist.

Status command751 is used to display the status of the copies of data as shown inFIG. 20 as working copy of data2006,2007 and2008. It will show the current status of each individual copy.

Branch command752 can cause a new working copy. For example, as shown inFIG. 20 new working copies2006,2007 or2008 can be created from the primary copy of data2005. This working copy is shown inFIG. 20 as working copy2006,2007 or2008. The token (e.g., the token inFIG. 1102) will be updated with the address of the new address to the copy data.

Checkout command753 can cause the working data (e.g., as shown inFIG. 202006,2007,2008) to be made accessible by causing a mount to occur using the mount information in the data token. The server and mount-naming options as shown in753 are used to override the attributes in the data token by specifying a different server and mount name than the attributes (e.g., as shown inFIGS. 3328 and329). The -u option as show in753 can cause the data to be dismounted when access is no longer needed.

Thecheckin command754 can cause a working copy (e.g., working copy2006 shown inFIG. 20) to be promoted to a primary copy2005. The primary copy2005 can, for example, have further branches (working copies2006 shown inFIG. 20) taken of it, andcheckout753 can be done by other users, etc.

The delete command755 will remove the working copy data (e.g., as shown as2006 inFIG. 20) from the copy data virtualization system as shown as104 inFIG. 1. The -f flag will force the operation rather than prompting the user to verify the command. If the data is mounted, it will be unmounted.

The createcommand756 will create a new primary copy (e.g., primary copy2005 shown inFIG. 20) data version using the information in the token. This can include anytransformation2004 processing, as discussed further herein. Along with creating a new primary copy, it will create a new copy data token (e.g., copy data token102 inFIG. 1). If -p option as show in756 is specified, the copy data token will be encrypted with the specified password.

Each of the commands inFIG. 7, 750, 751, 752, 753, 754, 755, other than create 756, can support an optional of -p to specify a password to access the token. The createcommand756 will use the password to encrypt the token. Each command can also support an option of the -t to use to specify a token file name. If it is not specified, an automatic name (e.g., “adk. dt,” where .dt represents the file is a data token) can be used. One of skill in the art can appreciate that while the present example is presented as a command line, it can easily be converted to an API or any other method used to communicate commands.

In some examples, the techniques provide a security model for the copy data tokens. The security for the copy data token can protect both the token and the data that a token provides access to.

In some examples, the security model can use one or more of the following security methods. First, the copy data token can be implemented as a file so any external encryption can be used to protect the token outside the scope of the copy data token environment. This can allow users to make choices and integrate with any existing security systems they are using. Second, the copy data token system has a built-in encryption that can be used to add a password to the token file. This is shown in the command line example onFIGS. 7, 750, 751, 752, 753, 754, 755 and 756. This can make the file readable and writeable only by the copy data token software. Third, each token file contains an encrypted checksum of the contents as shown inFIG. 3 aschecksum339. If the file is not encrypted but modified externally from the copy data token system, the checksum will not match the next time the software reads it and an error will be generated. Fourth, when a token is read and commands are executed based on the contents, the commands are sent to a Copy Data Virtualization system. The Copy Data Virtualization system contains access controls for the data and supports user, role and group based access controls.

FIG. 8 illustrates an exemplary computerized method for security processing for copy data tokens, according to some embodiments.FIG. 8 includes acopy data token843 and an application or a command line tool configured to use an API842 that communicates over anetwork844 with a copydata virtualization system845. The copydata virtualization system845 includes a list of unique copy data IDs847 and a list of access control entries846. The application or command tool using API842 is an example of software that supports command as shown inFIG. 7. Copy data token843 contains a set of attribute values, e.g., as show inFIG. 3.Network844 is an interconnect medium to allow software to communicate with other software. Copydata virtualization system845 can be a copy data virtualization system such as that described in U.S. Pat. No. 8,417,674, entitled “System and Method for Creating Deduplicated Copies of Data by Sending Difference Information Between Near-Neighbor Temporal States,” the contents of which are hereby incorporated by reference herein in their entirety. The list of unique copy data IDs that represent available primary copies of data847 is an internal list of the copydata virtualization system845 and contains a list of the primary copies of data (e.g.,FIG. 20 primary copy of data2005) that are available. List of access control entries for each primary copy of data along with user, role and group based access rights846 contains the access control information for people, software and data. The application or command line tool reads the copy data token843 and issues a command via the API842. The command can be one of the commands as show inFIG. 7. The command line tool842 adds the user credentials to the command information and sends the request overNetwork844 to the CopyData Virtualization System845. The CopyData Virtualization System845 has a list of unique Copy Data IDs847 and also a list of access control entries846 that provide access control options for the data, user, the role of the user and the groups of the user. The copydata virtualization system845 will make a decision to allow the command to be executed according to the information in the access controls846.

FIG. 9 illustrates an exemplary computerized method for security logic for a copy data token, according to some embodiments.FIG. 9 shows communication from theapplication948 to the copydata virtualization system949 and includes an operation to read acopy data token943, a decision to use a password901, an operation to decrypt the copy data token944, a decision if the checksum of the token is valid936, an operation to receive the success orfailure902. On the copydata virtualization side949,FIG. 9 shows an operation to receive anAPI message942, a decision to validate information903, a decision to checkuser access904, a decision if the command execution succeeds905, areturn success906 and returnfailure907.

The process starts with an operation to read thecopy data token943. This could be stored in a file in a JSON, XML or other format. The next step is to check if the user specified a password at decision point901. If the user did not specify a password, it will skip the next operation. If the user specified a password, execution will continue with decrypting the copy data token944. The decision is if the checksum (e.g., as show inFIG. 3 as checksum339) matches the value stored in the copy data token. If it does not match, a failure is return by continuing withoperation902. If it does match, execution will continue by combining the command and the credentials and sending it as shown at step945 to the copydata virtualization system949. Execution continues by receiving the message from the application atoperation942. The next step is to decide if the message information is valid. For example, does it contain a valid command as shown inFIG. 7. If the information is not valid, a failure will be returned throughoperation907. If the information is validated operation will continue to check if the user has access to the data specified in themessage904. If they do not have access, a failure will be returned throughoperation907. If they do have access to the data, the command will be executed and checked forsuccess905. If the command fails, a failure will be returned throughoperation907. If the command succeeds, a success will be return throughoperation906. When the return is received by the success orfailure operation902, it will return that information to the caller of the process.

FIG. 1 illustrates an exemplary creation process for a copy data token, according to some embodiments.FIG. 1 includes list of unique copy data IDs109 that represent available primary copies of data, a copydata virtualization system104, acomputer system105, a primary copy ofdata107, a working copy ofdata108, astorage network106 and acopy data token102. Copy data token102 is a copy data token as described in further detail inFIG. 3.

The copydata virtualization system104 is a system that can be configured to virtualize data. For example, the copydata virtualization system104 is a copy data virtualization system as described in U.S. Pat. No. 8,417,674, entitled “System and Method for Creating Deduplicated Copies of Data by Sending Difference Information Between Near-Neighbor Temporal States,” issued on Apr. 9, 2013, the contents of which are hereby incorporated by reference herein in their entirety. List of unique copy data ids109 is a list maintained by the copy data virtualization system of the available primary copy ofdata107 and working copy ofdata108. Thecomputer system105 creates an environment to allow software to be executed. Primary copy ofdata107 is data that has gone through a transformation process as described inFIG. 20,transformation process104. Working copy ofdata108 are copies that are created by the copydata virtualization system104 from the primary copy ofdata107.Storage network106 enables the copydata virtualization system104 andcomputer system105 to have access to storage containing copies of data. Copy data token102 encompasses the attributes as shown inFIG. 3. It is read by software running oncomputer system105 to help direct the software to specific working data copies108.

AComputer System105 will search the CopyData Virtualization System104 list of available copies of data109. After selecting acopy107, an instruction will be given to the CopyData Virtualization System104 to create a workingcopy108. The workingcopy108 will be created using storage virtualization techniques known to industry so that minimal actual storage would be used. This has a common name of a thin copy in the industry. The copy data token102 will have the reference to the workingcopy108. Software will use the token102 to make a request to the copydata virtualization system104 to create a working copy ofdata108.

FIG. 2 illustrates an exemplary computerized creation logic flow to create a copy data token, according to some embodiments.FIG. 2 includes aselection operation202, acopy operation204 and a copy datatoken generation operation206. Theselection operation202 will scan a list of available primary copy data (e.g.,FIG. 1, primary copy of data107) and return its id. Thecopy operation204 will cause the copy data virtualization system (e.g.,FIG. 1, copy data virtualization system104) to create a working copy ofdata108 from the primary copy ofdata107. The copydata virtualziation system104 will create a copy data token that describes the working copy ofdata108.

Atstep202, the copy data virtualization system selects an available copy data from the list of unique copy data IDs. For example, referring toFIG. 1 the copydata virtualization system504 selects an available copy data from the list of unique copy data IDs109. Atstep204, the copy data virtualization system initiates a copy of the selected copy data. Atstep206, the copy data virtualization system generates acopy data token102, which is described in further detail herein.

Below is an exemplary set of characteristics that a copy data token can contain. It can be implemented, for example, as an XML or JSON file. In the copy data token file, there can be multiple data sets as defined below. Each set of attributes will have a first attribute of “Begin [Name]311” attribute and a last attribute of “End313”. This allows more complex, dependent sets of data to be described by a single copy data token.

FIG. 4 illustrates exemplary attributes for a copy data token, according to some embodiments.FIG. 4 includes a copy data token attributes417,attribute name415 andattribute value416. Theattribute name415 provides a reference name that can be accessed by software. Theattribute value416 is the value associated with theattribute name415 and is the value returned to software when referencingattribute name415. The set is called a copy datatoken attribute417.

FIG. 4 shows a copy datatoken attribute417, which includes anattribute name415 and anattribute value416 for each copy data token attribute. In some embodiments, a copy data token is an organized collection of attributes and their associated values.FIG. 3 illustrates an exemplary organization of a copy data token.FIG. 4 illustrates a single copy datatoken attribute417 which is made up of anattribute name415, for example,FIG. 3,source address319, and the associatedattribute value416, for example, test. system. company. com/server1/disk12.

FIGS. 3A-3E, collectively referred to herein asFIG. 3, illustrate an exemplary copy data token, according to some embodiments.FIG. 3 includes copy data token310, begin section311,data set312, data setend313, beginsection361,data set362, data setend363,checksum339,attribute classes314, attributeclass source data315,tranformation316, user access317,access control318, attributesnames350, source data attributes351,source319,source address320, source access method321, source access protocol322, transformation attributes352, pre-processing323,subsetting324, masking325, data quality326, post-processing327, user access attributes353, requiredserver address328,server address329, mount naming330,access protocol331,access method332, copy data address333, copy data creation time334, access control attributes354,access control335, read/write336, expiration337, number users338 and checkin allowed341. A copy data token is a set of hierarchically organized copy data token attributes as shown inFIG. 4, copy datatoken attribute417. The hierarchy represented in the copy data token310 starts with a data set312 (in the example inFIG. 3, it is named “1”). The data set contains a set ofattribute classes314, forexample source data315. Each attribute class has a set ofattribute names350, for example the source data attributes351 is the set of attributes associated with thesource data class315. The source data attributes contain a set of attributes, for example,source319, which will contain a value of the description of the source or production data (e.g.,FIG. 20, production data2001).

A copy data token310 can use many different methods to store the set ofattribute names415 and attributevalues416 it contains. For this description, it will assume to be done with JSON (JavaScript Object Notation) which is well known in the industry. A single copy data token310 can hold the data formultiple Data Sets312 and362. EachData Set312 references a Working Copy ofdata108. This allows a single copy data token310 to reference complex and dependent data sets made up from multipleindividual Working Copies108 of data. TheData Set312 has 4 attribute classes and can be extended as needed. TheSource Data315 attribute class has the information that describes the original source of the data (e.g.,FIG. 20, production data2001). TheTransformation316 attribute class has the description of how the original data was modified to create thePrimary Copy107. User Access317 attribute class has the requisite information to provide the user with access to theWorking Copy108.Access Control318 attributes describe who can have access to the Working Copy of thedata108.Checksum339 is a hashcode of the contents of the token and is encrypted. Any time the token is read by the software, a new hashcode is generated and compared with thechecksum339 in the copy data token310 to see if the contents have been tampered with.

TheSource Data315 attribute class provides a description of the original source production data along with how to access it. This information is provided so the copy data token310 can be used to create thePrimary Copy107 from the original data along with being used as a method to referenceWorking Copy108 of data. Thesource attribute name319 contains a description of the source data. The information contained inSource Address320 is a unique ID109 that references one copy of data managed by the CopyData Virtualization System104. Source Access Method321 attributes can contain values such as Block, File System. It is used to inform software on the access method to use to read/write the data. This can be extended for other access methods. TheSource Access Protocol312 attribute describe the low level protocol use to read/write the data. This can contain values such as Fibre Channel, iSCSI, NFS or CIFS and can be extended to support other protocols.

TheTransformation316 attribute class contains information that can be used to describe how the original production data was modified to create thePrimary Copy107. There are three well known operations that are described—Subsetting324,Masking325 and Data Quality326. There are two other attributes used to capture any custom operations required—Pre-processing323 and Post-processing327.Subsetting324 is an operation that will copy a smaller portion of the original data into thePrimary Copy107. Masking325, also known as Data Masking in the industry modifies personal data so that it is not recognizable. For example, a credit card number is 4 sets of 4 digits. To allow theWorking Copy108 of the data to be used by people regardless of security rights, along with allowing software to work correctly which expects to see 4 sets of 4 digits, the value is modified using industry known approaches so the data stored in thePrimary Copy107 is no longer recognizable from the original. The Data Quality326 attributes is a broad set of operations that could be checking data for quality to correcting data. For example, it might check a field of data that is supposed to contain USA state abbreviations. There is a known set of values that can be here. This operation can check for them along with converting. An examples of converting would be to change Mass. as an abbreviation to MA. The Pre-processing323 attribute and Post-processing327 attribute describe operations that must be done before the well know operations or must be done after them. It can contain other transformations that are custom or set up and break down of special access requirements.

The User Access317 attribute class contains information for the user of the data to have access. The RequiredServer Address328 is an example of an attribute that cannot be modified once set during the initial creation of the token. If this is set to the address of a server, thenServer Address329 will be ignored. This allows the initial creator of the token to force the data to be mounted to a specific server for use. If this is blank,Server Address329 will be used as the address of the server to mount the data to. TheMount Naming330 attribute is used if the operating system on the RequiredServer Address328 or theServer Address329 supports the ability to name the mounted data.Access Method332 attributes can contain values such as Block or File System. It is used to inform software on the access method to use to read/write the data. This can be extended for other access methods. TheAccess Protocol331 attribute describe the low level protocol use to read/write the data. This can contain values such as Fibre Channel, iSCSI, NFS or CIFS and can be extended to support other protocols.Copy Data Address333 is aWorking Copy108 entry from the unique Copy Data ID109 list. The Copy Data Creation Time334 has the date and time theWorking Copy108 was created. This information describes where to get the data from and were to mount it to so the user can access the data along with which protocol to access it.

TheAccess Control318 attribute class contains information that was set during initial creation of the token and cannot be modified. TheAccess Control318 attribute class is used by the software to decide if a user has access to the data referenced by the copy data token310, User Access317 attribute class,Copy Data Address333. It Also provides information about how it will be set up and controlled by the CopyData Virtualization System104. Because access control information can be very large and complex, theAccess Control335 attribute contains an ID that is used by the CopyData Virtualization System504 to determine the access control that will be enforced. It will enforce access to a specific set of users, roles and groups. This can be expanded to provide other methods or classes of access. The Read/Write336 attribute will have a value of Read or Read Write. It informs the user if the data they are being given access to can be written to. The Expiration337 attribute contains the date when the data will expire and be deleted. The Number Users338 attribute will have the total number of mounts of the data referenced by the token. This would be set if performance is an issue. For example, it could be set to 1 which would mean the access to the data would not be shared. The Checkin Allowed341 attribute will let the user know if the checkin command is allowed for this data.

FIG. 5 illustrates an exemplary system for a copy data token system overview, according to some embodiments.FIG. 5 includes an application or command like tool using API501,network503, copydata virtualization system504,computer system505, copy data token502, working copy ofdata540 andstorage network506. The application or command line tool using API501 is software that will interpret the copy data token502 and send commands to the copydata virtualization system504. Thenetwork503 is used to allow software to communicate with other software. The copydata virtualization system504 is a copy data virtualization system as described in U.S. Pat. No. 8,417,674, entitled “System and Method for Creating Deduplicated Copies of Data by Sending Difference Information Between Near-Neighbor Temporal States,” issued on Apr. 9, 2013, the contents of which are hereby incorporated by reference herein in their entirety. The computer system allows software to be executed and can connect to data. Thestorage network506 allows computers to connect to data. The working copy ofdata540 is a copy created from primary copy of data2005. The copy data token is a set of attributes as shown inFIG. 3, copy data token310.

In this example, the copy data token is implemented as a file. The copy data token502 will have been acquired by copying it from a directory of available token files that have been published. This is only one of many methods that could be used to provide access to the available tokens. An application501 which can either be a copy data token specific command line tool or other software that will use an API to communicate usingNetwork503 with CopyData Virtualization System504. The application501 will read the information in the copy data token502 and pass necessary information to the CopyData Virtualization System504 to provide access to the data referenced by the token. The API communication would contain TheServer Address329, Mount Naming330 andCopy Data Address333, among other information needed to complete the operation. The CopyData Virtualization System504 will check theAccess Control318 attributes to decide if access to the data will be granted. If it is approved, The CopyData Virtualization System504 will mount theWorking Copy540, using theStorage Network506 to theComputer System505. A success will be return to the API and user will have access to the data fromComputer System505.

FIG. 6 illustrates an exemplary computerized method for a copy data token system overview logic flow, according to some embodiments.FIG. 6 includes anapplication601 side of the process, a copydata virtualization system604 side of the process, an operation to read a copy datatoken operation602, send command overnetwork operation610, success or failure receivedoperation611, receive api message fromapplication operation605, validate information send decision612, does user have access to thedata decision613, mount succeeds decision614, returnsuccess operation615, returnsuccess operation616.

The copy data token is a very powerful data service. It provides the basis for data sharing of information across many users, applications and needs without the need for expertise of the user. At the same time, the providers of the data need to retain necessary levels of security and integrity of the data. Note that although this document describes the operations done via a command line tool, they could also be accessed via an API for deeper integration. Below are some use case examples.

In one example, the copy data token can be used for product development and testing. The copy data token can provide the ability to access copies of production data without affecting the production systems. It can also be used in a manner which provides a self-service type of environment. On theapplication side601, the process starts by reading thecopy data token602. This provides an application with the information required to be able to access a working copy of data2006. The application will combine the information from the copy data token and send a command to the copydata virtualization system604. The receive api message fromapplication605 operation will pass the information to the validate information send decision612. This will check to make sure the information is correct, for example, checking to make sure the copy data address333 has a reference to existing working copy of data2006. If there is a problem it send failure to thereturn failure operation616. If the data is ok, it will flow to a check user hasaccess decision613. This will check the access control information to make sure the user has access to the specified data. If it fails, a failure will be sent to thereturn failure operation615. If it is ok, the mount command decision614 will execute. If the mount fails, a failure message will be sent to thereturn failure operation616. If it succeeds, a success message will be sent to thereturn success operation615. Thereturn success operation615 or thereturn failure operation616 will send a message back to the application which is received byoperation611 to see if the operation succeeded or failed. The outcome of the operation will be communicated back to the entity that initiated the operation.

FIG. 10 illustrates an exemplary computerized method for creating a copy data token for test and development environments, according to some embodiments.FIG. 10 includesoperation1020 to mount a copy of production data which after transformation will become the primary data,operation1021 to execute the transformation commands on the primary copy,operation1022 to use the command line tool andoperation1023 to show example set of command lines executed.

FIG. 10 is an exemplary computerized logic flow of creating a copy data token for test and development environments. Thefirst operation1020 is to request the copydata virtualization system104 to mount a copy of theproduction data2001. The end goal of the mount is to create a primary copy of data2005 that can then be used as the basis for working copy of data2006,2007 and2008.Operation1021 applies a set of transformation operations (e.g., as discussed herein). For example, Data Masking could be used with a goal of hiding real credit card numbers by replacing the numbers with fake ones. As other examples, the transform operations can include Data Subsetting, Data Quality Checking, and/or the like. The result of the transformation operations is data that can be provided into the development and test environments. When completed, the unique ID109 can be used to use in the token creation.

When thetransformation operation1021 is complete, the result becomes primary copy of data2005 and the unique ID is saved by the copydata virtualization system104 into the saved list of unique copy data ids109. The next step,operation1022 is to create a copy data token102 (e.g., using the edit command as shown inFIG. 7) and to add attributes.

Operation1023 can include a set of copy data token edit commands (e.g.,FIG. 7, edit command750). The edit commands can include, for example:

edit -n primary_data_set -f Source “Dev Test customer list data”

edit -n primary_data_set “Copy Data Address” “Unique_ID”109

edit -n primary_data_set Masking “Standard Corp Masking Rules”

edit -n primary_data_set “Access Method” Block

edit -n primary_data_set “Access Protocol” “Fibre Channel”

edit -n primary_data_set “Read/Write” “Read/Write”

The edit commands can load the information into theattributes417 in thecopy data token310. It can be seen that not all attributes must be specified and different implementations could have different defaults and different required attributes. At the end of theedit750 commands inoperation1023 ofFIG. 10, acopy data token310 has been created with asingle data set312 called primary_data_set. It has a description in thesource319 attribute of “Dev Test customer list data”. It references copy data address333 with Unique_ID109 that was gotten at the end ofoperation1021 inFIG. 10. Since there is noserver information328 and329, the server would need to be specified by the end user or application using the copy data token310 before causing acheckout753 command to be executed. The copy data token has information that theprimary copy107 will be mounted using Fibre Channel as a block device in a read/write manner. In some embodiments, the copy data token can be created using a JSON editor (e.g., if the copy data token was implemented as an JSON file).

FIG. 11 illustrates an exemplary system diagram for using copy data tokens in development and test environments, according to some embodiments.FIG. 11 includes a Git remote repository1160 which includessource code1174 and copy data token1175, an application or command line tool using API1163,network1164, copydata virtualization system1165,computer system1166, Gitlocal repository1168 containingsource code1167 and copy data token1169, primary copy ofdata1170, working copy ofdata1172, working copy ofdata1173 andstorage network1171. Git remote repository1160 is part of a Git source code or version control software system that is well known in the development and test industry. Application or command line tool using api1163 is any software that supports commands shown inFIG. 7 that can read acopy data token1169 and communicate with a copydata virtualization system1165. The copydata virtualization system1165 is a copy data virtualization system as described in U.S. Pat. No. 8,417,674, entitled “System and Method for Creating Deduplicated Copies of Data by Sending Difference Information Between Near-Neighbor Temporal States,” issued on Apr. 9, 2013.Network1164 provides the ability for the application1163 to communicate with copydata virtualization system1165. Thecomputer system1166 has the capability to run software and connect to data. The Gitlocal repository1168 is part of the Git source code or version control software system that is well known in the development and test industry.Source code1167 is managed by Git and stored in theGit repository1168. Copy data token1169 is of thetype310 and is stored along with thesource code1167. Primary copy ofdata1170 is an example ofFIG. 20 primary copy of data2005. Working copy ofdata1172 and1173 are examples of working copy of data ofFIG. 20 working copy of data2006,2007 and2008. Thestorage network1171 enables computers and software to connect with data.

Since development and test already have source code control systems in place to track versions of files, copy data tokens can be integrated into that environment to provide ease of use to the developers and testers. For this use case, Git (an open source product available at www.git-scm.com) will be used for exemplary purposes, according to a non-limiting embodiment. Git is a distributed revision control system with an emphasis on speed, data integrity, and support for distributed, non-linear workflows in the development and test environments.

The first step is to create the library of data and tokens that represent the data available (e.g., as discussed in conjunction withFIG. 1). This can be done by someone inside of IT or DBAs that have access to the production data copies in the copy data virtualization system. This can be accomplished using the copy data token create (e.g.,FIG. 7, create command756) command from the command line tool or the copy data can be created via some other means and the information would be manually added to a token using the edit (e.g.,FIG. 7, edit command750). In some embodiments as described herein the creation process can be automated or centrally controlled (e.g., by a copy data virtualization system).

For illustrative purposes only, the integration of the copy data token system into the source code control system will be described via a hook approach which is well known as part of the Git software. This approach can be applied to any number of methods of integration. In the case of the copy data token Administrator—the person creating the copy data—there is no integration to the source code control system. This is done so the initial adding of the copy data tokens to the control system does not create extra copies of data.

The organization of the token files across the source code control system can be dependent on the needs of the organization and users. They could be placed in each code set that makes sense to have access to a specific data copy. They could also be organized so each copy data token is in a separate area. Once the copy data token is in the developers or testers area, they will cause versions of data to be created. For the rest of this exemplary use case, it will be assumed that the copy data tokens are organized along with the product areas they make sense to be used in. Also, the exemplary description is for developers, but testers, etc. can follow a similar use path.

Below is an exemplary set of steps a developer would take and the automated operations that copy data tokens would cause via integration of hooks into the Git as the example system. One of skill can appreciate that one or more of the developer steps, and/or one or more of the steps below can be automated.

The first step a developer would do is a “git pull” from a remote repository. This allows the developer to see what is available. Before doing the checkout, the user may need to set themount server329 via a copy datatoken edit750 command. The next step is to execute a “git checkout” of source code from the Git system. There is an integration with Git which causes the copy datatoken branch752 command to be executed which creates a private copy of the data for the developer. A copy datatoken checkout753 command will also be executed to have it mounted and ready for use. Both of these commands occur as automation and are invisible to the user.

If the developer wants to try something they will do another “git branch”. This would cause another copy to be created by automation invoking the same commands (752 and753) that would be accessed within that particular branch. If the developer decides that their idea would not work they delete the branch. Not all commands in Git can have hooks defined for automated integration. If this is true, then the copy data token software will notice the change on its next integrated invocation and will execute the necessary commands. For example, deleting a branch in Git does not have a hook for integration. It is important for the copy data token software to keep track of the branches so it does not leave data existing that is no longer needed. In this case, on the next integrated invocation, for example for a checkout, the copy data software will look at the set of Git branches that exist. If there is data for a branch that no longer exists, it will unmount (if needed) and delete that data. Each new copy will also have an expiration set for them. If there is no later invocation of the copy data software to clean up left over data, they will automatically expire and be deleted.

The user has completed their work and executes a Git commit. This saves the changes on their local system. A hook is executed and the copy they received during checkout is now deleted.

Before the final commit, the developer could contact the copy data token Administrator to promote their working copy of data2006,2007 and2008 to a primary copy of data2005 for others to reuse. This would be done with thecheckin754 command. The last step is to either push the changes to the remote repository or the changes will get pulled.

For a more detailed description, the Git application has a remote repository1160 that is stored somewhere away from the user's local computer and storage. It includes source code files1162 along with the copy data token1161 created in theFIG. 10 logic flow. When the user executes a Git pull command, alocal repository1168 is created which is a copy of the source code files1162 in the remote repository. The complete set is copied to the local system and storage. When the Git pull commands is executed, a full copy of the repository, including thesource code1167 and copy data token1169 would exist in thelocal repository1168. The user would now have a private copy of the copy data token1169. The user would execute a status751 command to see if aServer Address329 attributes has been specified. If it is not specified, they would execute aedit750 command to set theServer Address329 attribute to thecomputer system1166 where they would like the workingcopy1172 to be mounted to.

The next step is to execute a Git checkout command. This creates a version of thesource code1167 along with the copy data token1169 that can be edited. The integration of the copy data token software with the Git software is accomplished with a hook. During the Git checkout command, the hook would execute abranch752 command which will create a workingcopy1172 from theprimary copy1170. The workingcopy1172 is a private copy of the primary copy ofdata1170. The hook would then execute acheckout753 command which would mount the workingcopy1172 using thestorage network1171 to the computer system1166 (which is the server named in the copy data token attribute Server Address329) for the user to read and write during development and testing.

A common Git command is branch. This is used to create a copy of the current copy of the source code, called a branch. The branch could then be used to try a code change without having to undo it in the case that it does not work and the branch is deleted. The Git branch command does not have hook for integration. The user must execute the Git checkout command after creating a new branch or before working on an older branch. There is an integration hook for the Git checkout command. When the Git checkout command is executed, the hook would execute the copy data software which will first check to make sure there are no changes with the branch structure of Git vs the copies that it has created. If a new copy is needed, abranch752 command which would create workingcopy1173 from theprimary copy1170. It would then execute acheckout753 command to mount it to thecomputer system1166 usingstorage network1171. If the new copy is not needed because it was created previously, then it will be checked for mount status and will acheckout753 command would be executed if needed. If the user decided that the change they was working on in the branch was not a good change, they would execute a Git branch delete command. There is no integration hook in Git, so the copy data token software will get invoked again on the next checkout. After analyzing the current Git branch structure, if it finds a branch no longer exists, it would execute delete755 command. The delete755 command will unmount the data if it is mounted tocomputer system1166, and delete the workingcopy1173.

When the user has completed their work, they will execute the Git commit command to tell the Git system that the current version should be saved. The hook will execute a delete755 command which will unmount and delete the working copy of data2006. The user has an option to decide to save the working copy of data and can execute acheckin754 command which would promote the working copy of data2006 to a primary copy of data2005.

Another exemplary use case is for application packages. There are many different ways to package one or more applications for distribution. For example, an application can be deployed as Virtual Machine (VM) and that could be packaged into an OVA (Open Virtualization Archive) file. Applications can be deployed is a “Zip” file which is a common name used for a file that contains compressed copies of other files. Applications can be deployed on CDs or DVDs with an executable copying files to the final destination. Applications can be downloaded with a local executable copying files from an external location to the local server. These are examples of the many methods used to deploy an application.

A new approach in the industry is an Application Container (i.e., Docker is an example of these new approaches and well known to the industry) which is a software container that holds the components necessary to deploy a complex application. It could have multiple application images along with their dependencies. One of the goals is to be able to move containers around quickly. If one of the applications requires a large data source such as a database, the size of the container could be very large. Another new approach is a vAPP from VMware which is a container of multiple Virtual Machines (VMs).

FIG. 12 illustrates an exemplary application packages, according to some embodiments.FIG. 12 includes theapplication package1280,application11281,application21282,application31283, configuration1284 which includescopy data tokens1285 and settings andsmall data1286, andpackage engine1287.

The application package1280 (e.g., as described previously) is a structure that contains one or more applications and other necessary information for deployment.Applications1281,1282 and1283 are examples and can be any software application. Configuration1284 contains other components necessary for the deployment of the applications. It can contain small amounts of data and configuration and set up information. TheCopy data token310 is added as part of the configuration information. Thepackage engine1287 may or may not be included with theapplication package1280. It is the software that interprets the application package to deploy it in a computer execution environment.

In some embodiments, these installation approaches copy the application and extra files to a location, configure the computer operating system as needed to run the application and are then complete. In some embodiments, some methods may have the same issue if the data required for the application(s) is too large. For example, one could get a software update over the internet measured in MBs but if the data update was measured in 100's of GBs or TBs, people look for better methods. For example, one way is to put the data onto a CD but it limited in capacity. Some embodiments could move to a DVD, then a blu-ray, tape or even a disk array, and/or the like.

In some embodiments, a problem can be to allow data to be deployed at the speed of applications that are part of application containers and enable the application containers to move between systems quickly while still retaining access to the data, regardless of the capacity of the data.

As the application container is deployed, the container engine, or script, is orchestrating the initiation of the applications along with setting up the resources needed. A plug-in to the container engine will enable a copy data token to be used to access data for any data environment as fast as the creation of applications regardless of the size of the data to be access.

In some embodiments, applications that requires access to pre-defined data, from IT or from an external source, can be set up during installation or during initial startup to use copy data tokens for access. One or more copy data tokens could be included with the installation. The token(s) can be modified during installation to customize to the target environment. For example, if the target was Linux VS Windows, the name of the mount point would be different. Either during the installation operation or during the initial application startup, the copy data tokens can be used to cause the correct data to be made available to the correct server.

The basic operation that will be used is thecheckout753 command. An extension that can be added to the container is to use thebranch752 command before the mount to have one or more (or all) application containers see a private copy of the data. When the container is shutdown, the delete755 command could be executed according to options chosen as part of the application container set up.

FIG. 12 illustrates an example of anapplication package1280. It contains threeapplications1281,1282,1283 and is not limited in numbers of applications. Each application would contain the dependencies that it needs such as binary libraries. Theapplication1281 could represent a virtual machine (VM) in which case it would also include an operating system as part of the virtual machine. Theapplication package1280 would also include apackage engine1287. This could be a script or a program that stays active even when the applications are executing. There is also configuration1284 information which include settings and small bits ofdata1286 along with thecopy data tokens1285. Thepackage engine1287 will read the configuration information1284 and deploy the applications (for example,1281,1282 and1283) contained in the package.

Thepackage engine1287 reads the configuration information and deploys each application. If an application has an associated copy data token1285 as called out in theconfiguration settings1286, then thepackage engine1287 would execute thecheckout753 command to cause the data to be mounted for the application to use. Optionally, thepackage engine1287 could execute abranch752 command first which could allowmultiple application packages1280 to be deployed and each application would have its own private copy of the data.

An alternative to this could be to place the copy data tokens inside the application which will then configure its own access to data at time of execution.

Another exemplary use case is using copy data tokens to provide copies of production data for analytics. In this case, the copies are created as previously but the copy data tokens are kept in single file system folder. Each user has access to the copy data token folder and can copy a copy data token for their processing. Once they have the set of copy data tokens, they would execute the copy data token branch command to create their own private copy of the data, followed by an copy data token checkout command to make it accessible for use. Since each user will have their own private copy, they can write custom results and/or changes to the data. If they made a mistake, or want to try something else, they can easily do another copy data token branch command followed by a checkout command. When they are done, they can either do copy data token delete commands or ask the Administrator to capture the results of their work to promote to a primary copy for others to use. This creates a very efficient environment to do analytics and try things without effecting other users. It also lets a single user to try things and quickly undo any changes.

FIG. 13 is an exemplary computerized method for creating a copy data token for analytics, according to some embodiments.FIG. 13 includesoperation1324 to mount a copy of the production data. When the operation is complete, this will become the primary copy of data,Operation1325 to execute the transformation,operation1326 to use the edit command andoperation1327 showing example edit commands.

FIG. 13 is an exemplary computerized logic flow of creating a copy data token for analytics environments. Thefirst operation1324 is to request the copydata virtualization system104 to mount a copy of theproduction data2001. The end goal is to create a primary copy of data2005 that can then be used as the basis for working copy of data2006,2007 and2008.Operation1325 applies a set of transformation operations as has been already discussed. For example, Data Masking could be used with a goal of hiding real credit card numbers by replacing the numbers with fake ones. When thetransformation operation1325 is complete, the result becomes primary copy of data2005 and the unique ID is saved by the copydata virtualization system104 into the saved list of unique copy data ids109. The next step,operation1326 is to use the edit command as shown inFIG. 7 to create acopy data token102 and add it attributes.Operation1327 can include a set of copy data token edit commands (e.g.,FIG. 7, edit command750). The edit commands can include, for example:

edit -n primary_data_set -f Source “Data for company warehouse”

edit -n primary_data_set “Copy Data Address” “Unique_ID”110

edit -n primary_data_set “Required Server Address” Server1

edit -n primary_data_set “Access Method” Block

edit -n primary_data_set “Access Protocol” iSCSI

edit -n primary_data_set “Read/Write” “Read/Write”

The edit commands shown will load the information into theattributes417 in thecopy data token310.

In some embodiments, not all attributes must be specified and different implementations could have different defaults and different required attributes. At the end of theedit750 commands inoperation1327 ofFIG. 13, acopy data token310 has been created with asingle data set312 called primary_data_set. It has a description in thesource319 attribute of “Data for company warehouse”. It references copy data address333 with Unique_ID110 that was gotten at the end ofoperation1325 inFIG. 13. Since there is noserver information328 and329, the server would need to be specified by the end user or application using the copy data token310 before causing acheckout753 command to be executed. The copy data token has information that theprimary copy107 will be mounted using iSCSI as a block device in a read/write manner.

FIG. 14 is an exemplary computerized method for using copy data tokens for analytics, according to some embodiments.FIG. 14 includes application or command line tool using API1490,network1491, copydata virtualization system1492,computer system1493, folder ofcopy data tokens1494, copy data token1495, primary copy ofdata1496, working copy ofdata1498 andstorage network1497. In some embodiments, the copy data token can be created using a JSON editor (e.g., if the copy data token was implemented as an JSON file).

Application or command line tool using API1490 is any software that supports commands shown inFIG. 7 that can read acopy data token1495 and communicate with a copydata virtualization system1492. The copydata virtualization system1492 is, for example, a copy data virtualization system as described in U.S. Pat. No. 8,417,674, entitled “System and Method for Creating Deduplicated Copies of Data by Sending Difference Information Between Near-Neighbor Temporal States,” issued on Apr. 9, 2013.Network1497 provides the ability for the application1490 to communicate with copydata virtualization system1492. Thecomputer system1493 has the capability to run software and connect to data. The folder ofcopy data tokens1494 is part of the can exist on any storage device that provides a directory or folder structure and allows uses to access the copy data token files. Copy data token1495 is of thetype310. Primary copy ofdata1496 is an example ofFIG. 20 primary copy of data2005. Working copy ofdata1498 is an example of working copy of data ofFIG. 20 working copy of data2006,2007 and2008. Thestorage network1497 enables computers and software to connect with data.

An IT Administrator or DBAs would go through the process of creating copy data tokens for each of the data sources to be shared with the Analytics team. The set ofcopy data tokens1495, in this example, are kept in a single folder. This could have any organization needed by the local group. A person working in Analytics will look through the copy data tokens available and select one or more. They would be copied to their local computer. They would then execute acheckout753 command which would mount theprimary copy1496 through thestorage network1497 to thecomputer system1493. As an option, they could have started with abranch752 command to create a workingcopy1498. This would allow them to make changes to the data and not affect any other users. When they were complete and if they wanted to keep the results, they could execute acheckin754 command which would promote the workingcopy1498 to aprimary copy1496.

Another exemplary use case for copy data tokens is for purchased research data. Many companies purchase research data from external sources or create research data to be used by other groups within the company. The problem they face is how to control the access, keep a log of activity and make sure no one can change the research data thereby affecting other people's research. By using copy data tokens to provide access to the data sources, the requirements are achieved. Each person will have a private copy of data so they can be free to modify it during the course of their research. When they have completed the research it will either be deleted manually or could expire automatically.

FIG. 15 is an exemplary computerized method for creating a copy data token for research data, according to some embodiments.FIG. 15 includesoperation1528 request to copy data virtualization system to create a new volume,operation1529 to use the edit command to create a copy data token andoperation1530 with example edit commands to create the copy data token.

FIG. 15 is an exemplary computerized logic flow of creating a copy data token for research data environments. Thefirst operation1528 is to request the copydata virtualization system104 to mount an empty volume similar to theproduction data2001 except it does not contain any information. The end goal is to create a primary copy of data2005 that can then be used as the basis for working copy of data2006,2007 and2008. The research data is copied into the empty volume and whenoperation1528 is complete, the result becomes primary copy of data2005 and the unique ID is saved by the copydata virtualization system104 into the saved list of unique copy data ids109. The next step,operation1529 is to use the edit command as shown inFIG. 7 to create acopy data token102.Operation1530 can include a set of copy data token edit commands (e.g.,FIG. 7, edit command750). The edit commands can load the information into theattributes417 in thecopy data token310. The edit commands can include, for example:

Edit -n research data -f Source “Research data for the research group”

edit -n research data “Copy Data Address” “Unique_ID”112

edit -n research data “Required Server Address” Server1

edit -n research data “Access Method” “File System”

edit -n research data “Access Protocol” iSCSI

Edit -n research data “Read/Write” “Read/Write”

In some embodiments, not all attributes must be specified and different implementations could have different defaults and different required attributes. At the end of theedit750 commands inoperation1530 ofFIG. 15, acopy data token310 has been created with asingle data set312 called research_data. It has a description in thesource319 attribute of “Research data for the research group”. It references copy data address333 with Unique_ID112 that was gotten at the end ofoperation1528 inFIG. 15. Since there is noserver information328 and329, the server would need to be specified by the end user or application using the copy data token310 before causing acheckout753 command to be executed. The copy data token has information that theprimary copy107 will be mounted using iSCSI as a file system in a read/write manner. In some embodiments, the copy data token can be created using a JSON editor (e.g., if the copy data token was implemented as an JSON file).

FIG. 16 is an exemplary computerized method for using copy data tokens for research data, according to some embodiments.FIG. 16 includes folder of script, copy data token andcommand line tool1600,script1604, application or command line tool using API1605, copy data token1608,network1601, copydata virtualization system1602,computer system1603, primary copy of data1606, working copy ofdata1609 andstorage network1607.

Application or command line tool using api1605 is any software that supports commands shown inFIG. 7 that can read acopy data token1608 and communicate with a copydata virtualization system1602. A folder of script, copy data token andcommand line tool1600 can be provided by any file system that provides a directory or folder structure. Thescript1604 can be implemented using any number of well-known scripting technologies such as Perl, Python, SH, Bash. It provide the execution of login taking a command line with arguments. The copy data token is of type copy data token310 inFIG. 3. The copydata virtualization system1602 is a copy data virtualization system as described in U.S. Pat. No. 8,417,674, entitled “System and Method for Creating Deduplicated Copies of Data by Sending Difference Information Between Near-Neighbor Temporal States,” issued on Apr. 9, 2013.Network1601 provides the ability for the application1605 to communicate with copydata virtualization system1602. Thecomputer system1603 has the capability to run software and connect to data. Primary copy of data1606 is an example ofFIG. 20 primary copy of data2005. Working copy ofdata1609 is an example of working copy of data ofFIG. 20 working copy of data2006,2007 and2008. Thestorage network1607 enables computers and software to connect with data.

Each of the researchers are given afolder1600 that contains ascript1604, a command line tool1605 and acopy data token1608. The script has two commands—start and stop. When the researcher executes the script with a command of start, it is executing the command line tool that executes abranch752 command. This command is sent to the CopyData Virtualization System1602 overNetwork1601. The CopyData Virtualization System1602 creates a workingcopy1609 from the primary copy1606. The next command in the script is acheckout753 command. The command is communicated to the CopyData Virtualization System1602 which mounts the workingcopy1609 to thecomputer system1603 usingstorage network1607. The researcher now has access to the research data in a private copy. When they are done, they execute thescript1604 with a command of stop. The script executes a delete755 command which is sent to the CopyData virtualization System1602. It dismounts the workingcopy1609 and deletes it.

Another exemplary use case for copy data tokens is supporting problem re-creation environments. One of the issues that is faced in IT is when a problem occurs in the production environment. IT cannot execute debugging on the production environment while it is running One approach to solving this is to create a problem re-creation environment. This could be the pre-production staging area or a separate area. The goal is to have an environment that is identical (or as close as possible) to production and then try to re-create the problem. Once this is done, debugging to find the root cause can take place. The problem faced by IT is capturing the application and data as close as possible to point the problem occurred. If one tried to use the last night backup, the data may not be a state that would cause the problem. A Copy Data Virtualization system has the capabilities of capturing both the application and data at any point during the day so it can be used right after the problem is detected. Once the data sets are captured, one needs a method to pass them to another team to do problem re-creation and make sure they have the correct versions of things needed. A single copy data token could be created that contains references to the data needed for a consistent problem re-creation environment.

In some embodiments, copy data tokens provide access to data. Applications can be stored on a media as data and look the same as any other data. For example, there could be a copy data token that has two data sets it references. The first data set is the application and the second data set is the database needed by the application. Once the application and data have been captured, a copy data token can be created either automatically by the software or manually via the command line. This will capture the correct data sets and makes it very easy to communicate this to the problem re-creation team. They get the copy data token.

One of the exemplary benefits of the copy data token is that it provides the ability to snapshot the environment. This means the problem re-creation team can create a copy of the captured copy, get instant access to it, and do debugging. The team does not need to worry if the debugging will destroying any state or data because they can instantly get another fresh copy. Once the team think they understand the problem, they can create a fix and apply it another instant copy and try it. Once they have decided on the correct fix, it can be applied for a final test. At the end of the session, they still have the original captured application, state and data if it is needed again.

FIG. 17 is an exemplary computerized method of problem re-creation for a copy data token, according to some embodiments.FIG. 17 includesoperation1731 to mount a copy of production data which will become the primary data,operation1732 to use edit command to create copy data token andoperation1733 to show example set of commands lines executed.

FIG. 17 is an exemplary computerized logic flow of creating a copy data token for problem re-creation environments. The first operation inFIG. 17, 1731 is to issue commands to the copydata virtualization system1812 to take a copy of theproduction data2001 which represented the production database and make a copy which becomes primary copy ofdata1815. Another command is sent to the copydata virtualization system1812 to take a copy of theproduction data2001 which represented the production application and make a copy which becomes primary copy ofdata1817.

Operation1732 can include a set of copy datatoken edit750 command to load specific attributes into the copy data token1814.Operation1733 represents an example of the edit command that can include, for example.

edit -n application -f Source “Billing Application”

edit -n application “Copy Data Address” “Unique_ID”200

edit -n application “Access Method” File System

edit -n application “Access Protocol” Fibre Channel

edit -n application “Read/Write” “Read/Write”

edit -n database Source “Billing Database”

edit -n database “Copy Data Address” “Unique_ID”201

edit -n database “Access Method” Block

edit -n database “Access Protocol” Fibre Channel

edit -n database “Read/Write” “Read/Write”

In some embodiments, not all attributes must be specified and different implementations could have different defaults and different required attributes. At the end of theedit750 commands inoperation1733 ofFIG. 17, a copy data token has been created with twodata sets312 called application and database. The first references acopy data address333 data withUnique_ID1600 that was gotten at the end ofoperation1731 inFIG. 17. This does not specify where the data will be mounted so it will need to be set before using the copy data token. The copy data token has information that the copy will be mounted using Fibre channel as a device and then mounted as a File System in a read/write manner. Thesecond data set312 is named database and has aUnique_ID1601 that was gotten at the end ofoperation1733 in FIG. e17. This does not specify where the data will be mounted so it will need to be set before using the copy data token. The copy data token has information that the copy will be mounted using Fibre channel as an operating system device and then mounted as a File System in a read/write manner. In some embodiments, the copy data token can be created using a JSON editor (e.g., if the copy data token was implemented as an JSON file).

FIG. 18 is an exemplary system diagram for problem re-creation for a copy data token, according to some embodiments.FIG. 18 includes application or command line tool using API1810,network1811, copydata virtualization system1812,computer system1813, copy data token1814, primary copy ofdata1815, primary copy ofdata1817, working copy ofdata1818, working copy of data1819, andstorage network1816.

Application or command line tool using api1810 is any software that supports commands shown inFIG. 7 that can read a copy data token1814 and communicate with a copydata virtualization system1812. Copy data token1814 is of thetype310 fromFIG. 3. The copydata virtualization system1812 is a copy data virtualization system as described in U.S. Pat. No. 8,417,674, entitled “System and Method for Creating Deduplicated Copies of Data by Sending Difference Information Between Near-Neighbor Temporal States,” issued on Apr. 9, 2013.Network1811 provides the ability for the application1810 to communicate with copydata virtualization system1812. Thecomputer system1813 has the capability to run software and connect to data. Primary copy ofdata1815 and1817 are examples ofFIG. 20 primary copy of data2005. Working copy ofdata1818 and1819 are examples of working copy of data ofFIG. 20 working copy of data2006,2007 and2008. Thestorage network1816 enables computers and software to connect with data.

The problem re-creation team has been given a single copy data token1814 which is small enough to be transferred as an attachment to an email. This copy data token1814 represents the production environment of the application and its database. Before working with the data, the command line1810 is used to execute anedit750 command to set theserver address329 for each of thedata sets312 which specifies where the data would be mounted to. The command line1810 is used to execute thebranch752 command which communicates the command to the CopyData Virtualization System1812 usingNetwork1811. The CopyData Virtualization System1812 will make a working copy ofdata1818 from theprimary copy1815 and a working copy of data1819 from theprimary copy1817. The command line1810 next is used to execute thecheckout753 command which will cause the workingcopy1818 and working copy1819 to be mounted to thecomputer system1813 usingstorage network1816. The full environment is now available for problem re-creation work. When the work is complete, the command line1810 will be used to send the delete755 command to the CopyData Virtualization System1812 which will unmount and delete the working copy ofdata1818 and working copy of data1819.

Another exemplary use case of using copy data tokens is for cloud environments. Cloud environments are well known in the industry as locations for computer and storage that are remote to the company or organization location. This use case is an example of archiving in the cloud.

In some embodiments, a very efficient archive can be created using Copy Data Virtualization. In this case, a capture of the data can be done on a schedule (for example once a day) and the changes can be replicated to the cloud. The efficiency is created because although there can be data that represents one copy a day, only the changes are stored for each day and storage virtualization is used to create the specific copy as it is needed. Being an archive, it is important that no one can change the base data.

As each new copy is created, a new copy data token310 is automatically created and named with date of creation. The user now has a library of tokens and can easily access the view of the data in the archive on any day that is needed. The user would be connected to the cloud environment. They would then use the command line or a special user interface that would display the available tokens. Once a token is selected, it would create a copy of the specific data and mount it to the specific server for access by the user. When they are done, they would delete the copy and the archive is still completely intact.

FIG. 19 is an exemplary system diagram for using copy data tokens for cloud archiving, according to some embodiments.FIG. 19 includes customer site based copydata virtualization system1941, application or command linetool using API1931,network1932, cloud based copydata virtualization system1933, cloud basedcomputer system1934, cloud folder ofcopy data tokens1939, copy data token1940, copy data token1935, primary copy ofdata1936, primary copy of data1938, working copy ofdata1942,storage network1937.

Customer site based copydata virtualization system1941 as described in U.S. Pat. No. 8,417,674, entitled “System and Method for Creating Deduplicated Copies of Data by Sending Difference Information Between Near-Neighbor Temporal States,” issued on Apr. 9, 2013. IT communicated with cloud based copydata virtualization system1933 to replicate copies of data from customer site to the cloud site. Application or command linetool using api1931 is any software that supports commands shown inFIG. 7 that can read acopy data token1940 and1935 and communicate with a copydata virtualization system1933. A cloud folder ofcopy data tokens1940 and1935 can be provided by any file system that provides a directory or folder structure. The copy data token is of type copy data token310 inFIG. 3. The copydata virtualization system1933 is a copy data virtualization system as described in U.S. Pat. No. 8,417,674, entitled “System and Method for Creating Deduplicated Copies of Data by Sending Difference Information Between Near-Neighbor Temporal States,” issued on Apr. 9, 2013.Network1932 provides the ability for theapplication1931 and customer site based copydata virtualization system1941 to communicate with copydata virtualization system1933. Thecomputer system1934 has the capability to run software and connect to data. Primary copy ofdata1936 and1938 are examples ofFIG. 20 primary copy of data2005. Working copy ofdata1942 is an example of working copy of data ofFIG. 20 working copy of data2006,2007 and2008. Thestorage network1937 enables computers and software to connect with data.

On a daily basis, the customer site copydata virtualization System1941 will capture changes from the customer environment and send them to cloud based copydata virtualization system1933 which is then saved intoprimary copy1936 onday 1 and primary copy1938 onday 2 and so on. Each time the copydata virtualization system1933 creates aprimary copy1936, it creates a matching copy data token1940 and1935. Over time, the cloud folder ofcopy data tokens1939 will contain one copy data token for each day of archive data under management by the cloud based copydata virtualization system1933. InFIG. 19,primary copy1936 has an associated copy data token1940 and primary copy1938 has an associated copy data token1935. Thecopy data tokens1940 and1935 are stored in acloud folder1939. When the user needs to gain access to the archived data in the cloud environment, they would connect to the cloud basedcomputer system1934 and execute thecommand line1931, with the appropriate copy data token from the cloud folder ofcopy data tokens1939, to send abranch753 command to the cloud based copydata virtualization system1933. This would cause the archived data stored in primary copy1936 (for example) to be copied to the working copy ofdata1942. The user would then execute acheckout753 command which would mount the working copy ofdata1942 to the cloud basedcomputer system1934 using thestorage network1937. They can access the archived data from the computer system. When they have completed the work, they would execute a delete755 command to dismount and delete the workingcopy1942.

The subject matter described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structural means disclosed in this specification and structural equivalents thereof, or in combinations of them. The subject matter described herein can be implemented as one or more computer program products, such as one or more computer programs tangibly embodied in an information carrier (e.g., in a machine readable storage device), or embodied in a propagated signal, for execution by, or to control the operation of, data processing apparatus (e.g., a programmable processor, a computer, or multiple computers). A computer program (also known as a program, software, software application, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file. A program can be stored in a portion of a file that holds other programs or data, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification, including the method steps of the subject matter described herein, can be performed by one or more programmable processors executing one or more computer programs to perform functions of the subject matter described herein by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus of the subject matter described herein can be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processor of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. Information carriers suitable for embodying computer program instructions and data include all forms of nonvolatile memory, including by way of example semiconductor memory devices, (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks, (e.g., internal hard disks or removable disks); magneto optical disks; and optical disks (e.g., CD and DVD disks). The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

To provide for interaction with a user, the subject matter described herein can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, (e.g., a mouse or a trackball), by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. For example, feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, speech, or tactile input.

The subject matter described herein can be implemented in a computing system that includes a back end component (e.g., a data server), a middleware component (e.g., an application server), or a front end component (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described herein), or any combination of such back end, middleware, and front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), e.g., the Internet.

It is to be understood that the disclosed subject matter is not limited in its application to the details of construction and to the arrangements of the components set forth in the following description or illustrated in the drawings. The disclosed subject matter is capable of other embodiments and of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting.

As such, those skilled in the art will appreciate that the conception, upon which this disclosure is based, may readily be utilized as a basis for the designing of other structures, methods, and systems for carrying out the several purposes of the disclosed subject matter. It is important, therefore, that the claims be regarded as including such equivalent constructions insofar as they do not depart from the spirit and scope of the disclosed subject matter.

Although the disclosed subject matter has been described and illustrated in the foregoing exemplary embodiments, it is understood that the present disclosure has been made only by way of example, and that numerous changes in the details of implementation of the disclosed subject matter may be made without departing from the spirit and scope of the disclosed subject matter, which is limited only by the claims which follow.

Claims (18)

The invention claimed is:

1. A computerized method for using a data token to automatically manage access to copy data associated with the data token, the data token defining attributes for the copy data associated with the data token to facilitate the automatic management of the copy data, the computerized method comprising:

storing, by a computing device with a processor and memory, the data token that defines attributes associated with copy data, wherein the copy data comprises both a full copy of data at a data source and one or more incremental changes to the data at the data source over time, the attributes comprising:

source data for the copy data indicative of the data source for the copy data;

transformation data for the copy data indicative of a set of modifications for the copy data specifying how the copy data is modified when creating an instance of the copy data; and

access data for the copy data indicative of:

a set of access settings for the copy data that define how an instance of the copy data is to be created for the user; and

a set of access permissions for the copy data that define an access level for the user for the copy data that defines how much of the copy data the user can access;

transmitting, by the computing device, the data token to a remote computer storing the copy data based on the source data in the data token; and

receiving, by the computing device, a customized copy of the copy data that the remote computer generated for the user from the copy data, the customized copy including modifications to the copy data based on the transformation data and the access data in the data token, thereby using the data token to automatically manage access to copy data associated with the data token for the user.

2. The computerized method ofclaim 1, wherein:

the copy data comprises a first subset and a second subset; and

the remote computer generates the customized copy of the copy data to comprise the first subset of the copy data and not the second subset of the copy data based on the transformation data.

3. The computerized method ofclaim 1, wherein the source data comprises at least one of:

a source address that defines how to access the copy data;

an access method that defines a method for accessing the copy data; and

an access protocol that defines a protocol to use to access the copy data.

4. The computerized method ofclaim 1, wherein the transformation data comprises at least one of the following:

a set of pre-processing parameters configured to protect security of information contained in the copy data;

a set of subsetting parameters configured to intelligently select a subset of the copy data if a full set of data in the copy data is not desired;

a set of masking parameters configured to mask one or more data fields in the copy data; and

a set of data quality parameters configured to degrade a quality of the copy data.

5. The computerized method ofclaim 1, wherein the set of access settings comprise at least one of the following:

a server address that specifies a server for the remote computer to mount the copy of the copy data;

an access method that defines a method for copying the copy data to the server; and

an access protocol that defines a protocol to use to copy the copy data to the server.

6. The computerized method ofclaim 1, wherein storing the data token further comprises encrypting the data token and storing the encrypted data token.

7. A computerized apparatus for using a data token to automatically manage access to copy data associated with the data token, the data token defining attributes for the copy data associated with the data token to facilitate the automatic management of the copy data, the apparatus comprising:

a processor configured to run a module stored in memory that is configured to cause the processor to:

store the data token that defines attributes associated with copy data, wherein the copy data comprises both a full copy of data at a data source and one or more incremental changes to the data at the data source over time, the attributes comprising:

source data for the copy data indicative of the data source for the copy data;

transformation data for the copy data indicative of a set of modifications for the copy data specifying how the copy data is modified when creating an instance of the copy data; and

access data for the copy data indicative of:

a set of access settings for the copy data that define how an instance of the copy data is to be created for the user; and

a set of access permissions for the copy data that define an access level for the user for the copy data that defines how much of the copy data the user can access;

transmit the data token to a remote computer storing the copy data based on the source data in the data token; and

receive a customized copy of the copy data that the remote computer generated for the user from the copy data, the customized copy including modifications to the copy data based on the transformation data and the access data in the data token, thereby using the data token to automatically manage access to copy data associated with the data token for the user.

8. The computerized apparatus ofclaim 7, wherein:

the copy data comprises a first subset and a second subset; and

the remote computer generates the customized copy of the copy data to comprise the first subset of the copy data and not the second subset of the copy data based on the transformation data.

9. The computerized apparatus ofclaim 8, wherein the transformation data comprises at least one of the following:

a set of pre-processing parameters configured to protect security of information contained in the copy data;

a set of subsetting parameters configured to intelligently select a subset of the copy data if a full set of data in the copy data is not desired;

a set of masking parameters configured to mask one or more data fields in the copy data; and

a set of data quality parameters configured to degrade a quality of the copy data.

10. The computerized apparatus ofclaim 7, wherein the source data comprises at least one-of the following:

a source address that defines how to access the copy data;

an access method that defines a method for accessing the copy data; and

an access protocol that defines a protocol to use to access the copy data.

11. The computerized apparatus ofclaim 7, wherein the set of access settings comprise at least one of the following:

a server address that specifies a server for the remote computer to mount the copy of the copy data;

an access method that defines a method for copying the copy data to the server; and

an access protocol that defines a protocol to use to copy the copy data to the server.

12. The computerized apparatus ofclaim 7, wherein storing the data token further comprises encrypting the data token and storing the encrypted data token.

13. A non-transitory computer readable medium having executable instructions operable to cause an apparatus to:

store a data token that defines attributes associated with copy data, wherein the copy data comprises both a full copy of data at a data source and one or more incremental changes to the data at the data source over time, the attributes comprising:

source data for the copy data indicative of the data source for the copy data;

transformation data for the copy data indicative of a set of modifications for the copy data specifying how the copy data is modified when creating an instance of the copy data; and

access data for the copy data indicative of:

a set of access settings for the copy data that define how an instance of the copy data is to be created for the user; and

a set of access permissions for the copy data that define an access level for the user for the copy data that defines how much of the copy data the user can access;

transmit the data token to a remote computer storing the copy data based on the source data in the data token; and

receive a customized copy of the copy data that the remote computer generated for the user from the copy data, the customized copy including modifications to the copy data based on the transformation data and the access data in the data token, thereby using the data token to automatically manage access to copy data associated with the data token for the user.

14. The non-transitory computer readable medium ofclaim 13, wherein:

the copy data comprises a first subset and a second subset; and

the remote computer generates the customized copy of the copy data to comprise the first subset of the copy data and not the second subset of the copy data based on the transformation data.

15. The non-transitory computer readable medium ofclaim 14, wherein the transformation data comprises at least one of the following:

a set of pre-processing parameters configured to protect security of information contained in the copy data;

a set of subsetting parameters configured to intelligently select a subset of the copy data if a full set of data in the copy data is not desired;

a set of masking parameters configured to mask one or more data fields in the copy data; and

a set of data quality parameters configured to degrade a quality of the copy data.

16. The non-transitory computer readable medium ofclaim 13, wherein the source data comprises at least one of the following:

a source address that defines how to access the copy data;

an access method that defines a method for accessing the copy data; and

an access protocol that defines a protocol to use to access the copy data.

17. The non-transitory computer readable medium ofclaim 13, wherein the set of access settings comprise at least one-of the following:

a server address that specifies a server for the remote computer to mount the copy of the copy data;

an access method that defines a method for copying the copy data to the server; and

an access protocol that defines a protocol to use to copy the copy data to the server.

18. The non-transitory computer readable medium ofclaim 13, wherein storing the data token further comprises encrypting the data token and storing the encrypted data token.

Images