US20030126144A1 - Method and apparatus for access, integration, and analysis of heterogeneous data sources via the manipulation of metadata objects - Google Patents

Abstract

A computer-based method represents a subset of a dataset table of rows and columns. The method includes selecting a set of blocking variables corresponding to blocking columns of the dataset table. For each row in the dataset table, a tuple of values for the blocking columns uniquely identifies the row within the dataset table. The method also includes selecting a set of non-blocking variables that correspond to columns of the subset. The set of non-blocking variables does not intersect the set of blocking variables. The method also includes creating a block information structure that includes both the set of non-blocking variables and, for each blocking variable in the set of blocking variables, a set of values.

Description

CLAIM OF PRIORITY
• This application claims priority under 35 USC §[0001]119(e) to U.S. Patent Application Serial No. 60/311,495, filed on Aug.10,2001, the entire contents of which are hereby incorporated by reference.
TECHNICAL FIELD
• This invention relates to computer information systems, and more particularly to the storage and manipulation of metadata for data sources.[0002]
BACKGROUND
• In a complex technological environment, information is commonly stored in heterogeneous databases on a wide range of media. An organization's mission-critical information can be housed in a central server, updated on a continual basis via an online transaction processing (OLTP) system or an enterprise resource planning (ERP) system, using relational databases like Oracle, Sybase, or Microsoft Access. Other information can be housed remotely on servers and downloaded via specialized software. Still other information can be stored on Compact Discs and updated periodically with new releases.[0003]
• One traditional approach to integrating disparate data sources is built around the notion of a “data cube,” or dimensional database, which employs a star schema to organize the constituent data sets. This technology can place high demands on system resources when the entire data cube must be rebuilt after a single data set or data point changes. Also, since the entire data cube must be traversed when merging, subsetting, or analyzing data, this process can be slow, creating system bottlenecks.[0004]
SUMMARY
• In general, in one aspect, the invention features a computer-based method of representing a subset of a dataset table of rows and columns. The method includes selecting a set of blocking variables corresponding to blocking columns of the dataset table. For each row in the dataset table, a tuple of values for the blocking columns uniquely identifies the row within the dataset table. The method also includes selecting a set of non-blocking variables that correspond to columns of the subset. The set of non-blocking variables does not intersect the set of blocking variables. The method also includes creating a block information structure that includes both the set of non-blocking variables and, for each blocking variable in the set of blocking variables, a set of values.[0005]
• Preferred embodiments include the following. For each row in the subset that has a tuple of values for the blocking columns, the values of the tuple are included in the corresponding sets of blocking-variable values. The subset of the dataset table includes the entire dataset table. The number of elements in the set of non-blocking columns, plus the number of elements in the sets of values for the blocking columns, is linearly proportionate to an upper bound on the binary storage requirements of the block information structure, particularly when such sets are arbitrarily large. The block information structure is stored on a machine-readable medium.[0006]
• Among other advantages, this aspect of the invention provides a method for combining, selecting, and delivering data from heterogeneous databases. The method allows for the creation and manipulation of metadata entities called blocksets. Blocksets contain summary information about sets of data, and can therefore be manipulated quickly, flexibly, and efficiently in place of the datasets themselves. Blocksets contain metadata about the datasets, allowing a user to choose combinations of the datasets for viewing without having to access the datasets directly.[0007]
• In general, in another aspect, the invention features a computer-based method of accessing information in heterogeneous databases. The method includes presenting a graphical user interface, with controls representing a data cart and a plurality of datasets. The method also includes receiving user input that selects a dataset to add to the data cart. The method also includes generating a block information structure that specifies the dataset, and adding the block information structure to the data cart.[0008]
• Preferred embodiments include the following. The method incorporates into the block information structure a set of non-blocking variables, a set of blocking variables, and for each such blocking variable, a set of values. The dataset includes a plurality of rows, each identified by a corresponding tuple of values from the sets of values for the blocking variables. For a blocking variable in the set of blocking variables, the method further includes presenting enumeration controls in the graphical user interface. Each enumeration control corresponds to an existing value in the dataset for the blocking variable. The method further includes collecting user input that specifies a subset of the dataset, and includes representing the subset in corresponding block information structure. The method further includes saving the data cart to a persistent storage medium. The method further includes adding a second block information structure to the data cart, in response to user input. The controls representing a data cart include a symbol of a shopping cart.[0009]
• The graphical user interface allows a user to construct blocksets known as data carts. The method further includes collaboration features such as the ability to save and comment on blocksets, and the option of peer-to-peer systems that accommodate geographically dispersed data sources.[0010]
• In general, in still another aspect, the invention features a computer-based method of retrieving information represented by a blockset. The method includes connecting to databases, wherein each database corresponds to a block in the blockset. Each such block specifies a subset of a dataset stored in the corresponding database. The blockset has a plurality of blocking variables. The blocks each include a set of non-blocking variables, and have a set of values for each blocking variable in the plurality of blocking variables. The method includes adding a blocking column to a derived table, once for each blocking variable in the set of blocking variables. The method also includes adding to the derived table a non-blocking column for each element in a union of the non-blocking variables in the plurality of blocks. Furthermore, the method includes adding a row to the derived table. The row includes a cell for each column in the derived table. The row is uniquely identified by a tuple of values from the sets of values for the blocking variables. The method also includes populating a cell of a non-blocking column in the row, using a value retrieved from the database corresponding to the block. The block contains the non-blocking variable corresponding to the non-blocking column.[0011]
• Preferred embodiments include the following. The method further includes adding a row for each tuple of values from the sets of values for the blocking variables, provided the tuple occurs in at least one dataset corresponding to a block in the plurality of blocks. The method includes, when connecting, using each block as a basis for a database query that specifies the corresponding subset. The database query uses Structured Query Language.[0012]
• In general, in yet another aspect, the invention features a computer-based method of representing a table derived from a blockset, including outputting blockset metadata that describes the blockset. The blockset metadata includes fields for a blockset title and a blockset description. The method also includes outputting column metadata for a column in the table, such that the column metadata describes a variable associated with the column. The variable is associated with an underlying dataset that provides data to the table in the blockset. The column metadata includes fields for a title of the variable and for a title of the underlying dataset.[0013]
• In general, in another aspect still, the invention features a computer-based method of collecting metadata for a dataset, including prompting a user to provide a database name. The method also includes confirming that the database name represents a database, displaying a list of tables in the database to the user, receiving user input specifying a table in the list of tables, and prompting the user to confirm that a list of blocking variables and a list of non-blocking variables are correct for the database. In addition, the method includes prompting the user to confirm metadata for the dataset and for the list of non-blocking variables. If the user confirms the list of blocking variables, the list of non-blocking variables, and the metadata, the method also includes adding a dataset corresponding to the table to a collection of datasets.[0014]
• Preferred embodiments include the following. The metadata includes a title for the dataset. The metadata includes a description for the dataset. The metadata includes a title for a non-blocking variable in the list of non-blocking variables. The metadata includes a description for a non-blocking variable in the list of non-blocking variables.[0015]
• The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.[0016]
DESCRIPTION OF DRAWINGS
• FIG. 1A is a schematic diagram of an information processing system.[0017]
• FIG. 1B is a schematic diagram of physical components of an information processing system.[0018]
• FIG. 1C is a diagram of a database schema for a data group example.[0019]
• FIG. 2 is a diagram is a schematic diagram of relationships between datasets, blocks, blocksets, and blockset derived tables.[0020]
• FIG. 3A is a diagram of block creation from datasets.[0021]
• FIG. 3B is a formula expressing a block.[0022]
• FIG. 4 is a schematic diagram of relationships between blocks, blocksets, and blockset derived tables, including n-tuples.[0023]
• FIG. 5A is a flowchart of a derivation process.[0024]
• FIG. 5B is a formula for a database query to retrieve a dataset table underlying a block.[0025]
• FIG. 5C is a codebook table.[0026]
• FIG. 6 is a snapshot of a user interface for a login page.[0027]
• FIGS. 7A through 7E are snapshots of a user interface for a browsing environment.[0028]
• FIGS. 8A and 8B are snapshots of a user interface for a data cart.[0029]
• FIG. 9 is a snapshot of a user interface for a sample codebook.[0030]
• FIG. 10 is a snapshot of a user interface for a data cart information entry page.[0031]
• FIGS. 11A and 11B are snapshots of a user interface for a dataset archive.[0032]
• FIGS. 12A through 12F are snapshots of a user interface for an administrator page.[0033]
• FIG. 12G is a flowchart for an upload process.[0034]
• FIG. 12H is a diagram of a database schema for a codebook table.[0035]
• FIG. 13 is a flowchart of a codebook process.[0036]
DETAILED DESCRIPTION
• Referring to FIG. 1A, physical elements of an[0037]information processing system10 include aserver12 connected toclient machines14 such as workstations, laptops, or desktop computers via anetwork16. For example,network16 could include an Internet connection, an intranet within a single organization, or an extranet connecting an organization and its associates.
• [0038]Server12 includes software components such as aweb server engine12a,databases18, and anapplication database20.Web server engine12aserves outweb pages12bover the network.Web server12aconnects to database tables18aofdatabases18 via a database connection protocol, such as ODBC (Open DataBase Connectivity)12c.Web server12aalso connects toapplication database20, which includes metadata tables and user information tables. Furthermore,web server engine12aencodes procedures for creating and manipulating metadata objects called blocksets and related entities, as will be explained in more detail with regard to FIG. 1C.
• A user[0039]22 interacts withinformation processing system10 viaclient machine14.
• Referring to FIG. 1B,[0040]client machine14 includes aprocessor14athat operates on data stored inmemory14band executes machine-readable instructions stored instorage14eand inmemory14b.Client machine14 renders visual information on adisplay device14cand receives input from a user22 via one ormore input devices14d, such as a mouse and keyboard.Storage14eincludes an operating system that uses a point-and-click GUI.Client machine14 also runs a web browser application, not shown, that logically connects toweb server engine12a(FIG. 1A).Network interface14fconnectsstandalone device14 tonetwork16. Bus14gcarries information between components ofstandalone device14.
• Components of[0041]server12 are comparable in general structure and function to their like-named counterparts inclient machine14. In particular,processor12hexecutes machine-readable instructions that encodeweb server engine12a,databases18, andapplication database20.
• A[0042]human administrator22ainteracts withserver12.Administrator22atypically has advanced privileges onserver12, while typical users22 do not. In this discussion, however, unless specified otherwise, the term “user” refers to both users22 and toadministrators22a. That is,administrators22aare a subset of users22.
• Referring to FIG. 2, logical entities used in[0043]system10 include adataset24, ablock26, ablockset28, and a blockset derived table30. Broadly, and as will be explained in more detail, adataset24 represents a table of data. Ablock26 summarizes a portion of a dataset (including perhaps the entire dataset); the summarized dataset is said to “underlie” the block. Ablockset28 collects one or more blocks. A blockset derived table30 is a table of data for a blockset, collecting, for the blocks in the blockset, the corresponding portions of the underlying datasets.
• Dataset[0044]
• Referring to FIG. 3A,[0045]dataset24 is an information structure that encodes a series of observations on a given set of variables. The observations are stored indata sources18a(FIG. 1).
• This description refers to elements of[0046]dataset24 in the standard manner, with the data arranged in a table whosecolumns24arepresent variables and whose rows24brepresent observations. For a variable i that includes an M by 1 column vector Xi, where “M” is the number of rows, then an M by N dataset is a matrix of the form [X₁X₂. . . X_n]. This matrix contains blockingvariables24cwhich, collectively, uniquely identify a given observation. Other variables in the dataset arenon-blocking variables24d. For instance, a dataset on cars might uniquely identify each car by its make, model and year. Make, model and year would be encoded as blocking variables of the corresponding dataset. Non-blocking variables might include the car's size, price and gas mileage. As another example, a financial dataset might identify stocks by the ticker symbol and the day from which the closing price was taken. The non-blocking variables could include the stock price, volume traded, and any splits that might have occurred on that day. Each dataset contains at least one non-blocking variable.

  	TABLE 1
  	WorldInfo dataset definition and metadata:
  	DsetID = 345;
  	DsetBlockingVars = Year, Country;
  	DsetNonblockingVars = Party, Tenure, ElexnMode;
  	DsetTitle = “Political Indicators”;
  	DsetDatabaseName = PolInd.mdb;
  	DsetTableName = “Indicators Data”;
  	DsetCodebookName = PolIndCodebook;
  	DsetDescription = “Various political indicators from IFC”;
  	DsetCategoryID = 3;
  	DsetCitation = “IFC Annual Report, 1998”.

• In the example of Table 1, the necessary metadata elements are the database name, the blocking variable list, and the non-blocking variable list. The other metadata are optional, and may be used to describe the data set in the user interface, described with regard to FIGS. 6 through 12F, below. Other corresponding tables are used in conjunction with the dataset metadata, such as a Variables table with variable descriptions, a Categories table with textual descriptions of the category codes, or Authors and Citations tables with more extensive bibliographic information.[0047]
• Data Group[0048]
• A[0049]data group34 is a collection ofdatasets24 having identical blocking variables. Typically, datasets within a data group are logically connected to each other by referencing the same basic type of data. For instance, in the “WorldInfo” example of Table 1, all datasets have blocking variables for Country and Year. Thus they form a data group describing countries' economic, political, demographic, social, and geographic conditions on a year-to-year basis. Note that a single data set could logically belong to more than one group; for example, a data set containing daily stock prices could also belong to a group of yearly corporate data, simply by averaging daily information over each one-year period. Furthermore, one practiced in the art would recognize that various data groups can be combined together, for instance via the use of composite primary keys and intermediary translation tables.
• Database Example[0050]
• Referring now to FIG. 1C, an[0051]example application database20 encodes asingle data group34 conforming to the “WorldInfo” example. That is, the blocking variables for each dataset are Country and Year.
• [0052]Application database20 includes a dataset table36, a variable table38, an author table40, an author-dataset table40p, a category table42, a category-dataset table42f, a user table44, a block table46, and a blockset table48.
• Dataset table[0053]36 stores information for adataset24. Each row of dataset table36 includes a dataset key36a, which uniquely identifies rows within dataset table36. Such a row also includes metadata fields for presentation to a user, such as a title36b, which can be used as a caption in a user interface; description36f, which can storephrases describing dataset24 in detail; and codebook name36g. Each row also includes metadata for locating thedataset24 in adata source18a, such as a table name36cand a database name36d. Fields for variable list36k,country list36m, and year list36nstore comma-delimited lists of values from the correspondingdata source18a. These lists cache data that specifies a block, reducing the need to connect directly todata source18a. In particular,country list36 stores the distinct values in the underlying data for the blocking variable Country in the WorldInfo example. Similarly, year list36nstores distinct values for Year. Other bookkeeping data in dataset table36 includes a first version36eand acreation date36p.
• Variable table[0054]38 stores information about non-blocking variables associated with adataset24. Each row of variable table38 describes a variable. One row of dataset table36 can be associated with many rows of variable table38 via the foreign key fieldvariable dataset ID38b. Each row includes a title38cfor the variable, aunits notation38fof the values the variable measures, a list of keywords36dassociated with variable for searching, and a description36e, which can store phrases describing the variable in detail. Name38gindicates the corresponding column name for the variable in the underlying table. Rows also include variable key38a, which uniquely identifies the rows within variable table38.
• Author table[0055]40 and category table42 provide supplementary metadata for adataset24, allowing many-to-many relationships between dataset table36 and author table40, and between dataset table36 and category table42. In general, a given author can be associated with one or more datasets, and vice versa. Likewise, categories can be used to describe one or more datasets, and a given dataset can belong to one or more categories.
• Each row of author table[0056]40 represents an author that can be associated with data in adataset24. Each such row includes fields for afirst name40b, alast name40c, anemail address40d, awebsite address40e, and anaffiliation40f. Rows also include an author key38a, which uniquely identifies the rows within author table40.
• Author-dataset table[0057]40pimplements the many-to-many relationship between dataset table36 and author table40.Link key40quniquely identifies the rows within author-dataset table40p.Author FK40ris a foreign key referencing author key40a.Dataset FK40sis a foreign key referencing dataset key36a.
• Each row of category table[0058]42 represents a category that can be associated with data in adataset24. Rows include a category key42a, which uniquely identifies the rows within author table40. Each such row includes fields for acategory name42band a description42d, which can store phrases describing the category in detail. A field for parent category42cis a reflexive foreign key, i.e., one that references category key42aof its own table, allowing category table42 to store nested hierarchies of categories.
• Category-dataset table[0059]42fimplements the many-to-many relationship between dataset table36 and category table42. Link key42guniquely identifies rows within author-dataset table42f. Category FK42his a foreign key referencing category key42a. Dataset FK42kis a foreign key referencing dataset key36a.
• Each row of user table[0060]44 stores persistent information about a system account for a human user22, including a user name44b, alogin name44c, a password44d, and admin level44m, which specifies a level of system privileges granted to the user. Each row also stores metadata about the user, such astitle44e, anemail address44f, awebsite44g, and adescription44h. A field foraffiliation44kindicates an organization or domain for the user.Latest dataset44nindicates thedataset24 last used by the user, allowing the user to return to this dataset in a subsequent session.
• Each row of block table[0061]46 represents a block26 (FIG. 2). One row of dataset table36 can be associated with many rows of block table46 via the foreign key field dataset FK46b. Conversely, one row of block table46 references one row of dataset table36. That is, eachblock26 has onedataset24, but onedataset24 can havemany blocks26.
• One row of blockset table[0062]48 can be associated with many rows of block table46 via the foreign keyfield blockset FK46c. Eachblock26 has oneblockset28, but oneblockset28 can havemany blocks26.
• Each row of block table[0063]46 in the described example has alist46dof non-blocking variables associated with theblock26. Each row also has a set of values for the blocking variables Country and Year in thefields countries46eandyears46f, respectively. Rows also include a block key46a, which uniquely identifies the rows within block table46.
• Each row of blockset table[0064]48 represents ablockset28. Each row has metadata describing theblockset28 for presentation to a user, including fields fortitle48e,description48c,authors48m,citations48n, andcategories48p. Each row also includes bookkeeping fields such as date48d,version48g, published48h, submitted48k, and deleted48q. A foreign key field user FK48breferences a row in user table44, indicating the user22 who owns theblockset28. A field for public48findicates whether theblockset28 should be shared with other users. Rows also include a blockset key48a, which uniquely identifies the rows within blockset table48.
• In the WorldInfo example, a country table, not shown, maps three-letter abbreviations to country names. For example, an entry in the table contains an abbreviation value of “ITA” and a country name value of “Italy”.[0065]
• Blocks[0066]
• In general, blocks refer to sets of data via metadata information. A block summarizes a set of data by including blocking-variable values that specify the set of data, together with metadata about additional, non-blocking-variable columns to associate with the block.[0067]
• Referring to FIG. 3A, a[0068]block26 describes a subset of data from onedataset24. The subset is defined by specifying a set ofnon-blocking variables24dfrom the dataset and a set of values from each of the blockingvariables24c. Thus a block B from a dataset having m blocking variables and n non-blocking variables is represented by a (m+1)-tuple, shown in FIG. 3B. The V_iare non-blocking variables and the X₁are legal subsets of values from the m blocking variables. For the example WorldInfo implementation, a corresponding block definition includes: [{Non-blocking vars}, Country set, Year set].

  	TABLE 2
  	WorldInfo block definition:
  	VariableSet = {23, 45, 73, 215}.
  	CountrySet = {GER, USA, SWE, AUS, CAN, JAP};
  	YearSet = {1980, 1985, 1990, 1995, 2000};

• Table 2 gives a sample block definition for the WorldInfo example. The specified values for the Country blocking variable are codes for Germany, USA, Sweden, Austria, Canada, and Japan. The specified values for the Year blocking variable are 1980, 1985, 1990, 1995, 2000. Non-blocking variable are referred to by their key values[0069]38a(FIG. 1C).
• Referring again to FIG. 2, as an example of a process of constructing a block, block[0070]26 can represent anentire dataset24. In this case,dataset24 contains blockingvariables24clabeled BV1 and BV2 andnon-blocking variables24dlabeled NBV1 and NBV2. Theblock26 will therefore include thevalues26athat BV1 and BV2 take on within the desired subset. For instance, a year set might consist of the set {1991, 1992, 1993, 1994, 1995, 1996}. Theblock26 also includes the names or labels of the non-blocking variables NBV1 and NBV2, which are stored in a list infield26b. Notice that the names themselves or pointers to name identification fields are sufficient; the actual data within these fields need not be stored in the block itself.
• Blocks require relatively little storage memory to reference large collections of data. For instance, a block containing i variables for j countries and k years can reference as many as i*j*k rows from the corresponding dataset, but the block can describe these rows using only i+j+k elements—a significant improvement for large values of i, j, or k. In general, the number of elements in the set of non-blocking variables and in the sets of values for the blocking columns provides an approximate upper bound on binary storage requirements of the block, subject to a linear scaling factor. As the sets grow arbitrarily large, the binary storage requirements of the block are on the order of the sum of the sizes of these sets. In other words, the binary storage requirements are linearly proportionate to the cardinality of these sets.[0071]
• Blocksets[0072]
• Referring to FIGS. 2 and 4, a[0073]blockset28 is an unordered set of one ormore blocks26 from a single data group. For every block, there is a corresponding simple blockset with only one block. A user22 interacts withsystem10 in the context of one ormore sessions52 onclient machine14. Operations in onesession52 are independent ofother sessions52. A data cart50 is a blockset28 corresponding to the set of blocks currently selected in a session. Each session has either zero or one data cart50. When initialized, a data cart is empty.

  	TABLE 3
  	WorldInfo blockset definition and metadata:
  	MyBlocks = {43, 44, 45};
  	CreatorId = “joseph42”;
  	Title = “Political Economy Dataset”;
  	Description = “War, GDP, and famine data for Africa”;
  	Version = “1.3”;
  	Submitted = “Aug. 4, 2001”;
  	Authors = “Joseph Kalt”;
  	Categories = “Politics, Economics, Development”;
  	IsPublic = true.

• Of the fields in the example of Table 3, only the first (MyBlocks, expressing the collection of blocks) is strictly necessary to define the blockset; all others are examples of blockset metadata that is useful in storage and user display functions. In particular, keeping track of the blockset creator in the creatorId field allows blocksets to be saved and shared with other users.[0074]
• Since blocksets include collections of blocks, in general, they place minimal demands on memory. Furthermore, in[0075]system10, ablockset28 is encoded in software as ablockset object54. Thus, blockset objects contain bothproperties54aandmethods54b; that is, data and operations.Properties54ainclude theset54cof blocks associated with theblockset28, such as the vector myBlocks in the example of Table 3. Blockset objects54 can also possessmethods54bwhich contain instructions on how to perform certain operations on blockset objects54. For instance, ablockset object54 can display summary information about itself by counting the number of variable entries in each of its constituent blocks. It can also associate a new block to itself by adding a reference to that block to itscollection54c. Ablockset object54 can merge itself with anotherblockset object54 via a set union operation.
• Blockset Derived Tables[0076]
• [0077]System10 can download blockset data to a client machine via a number of standard output formats.
• Referring to FIG. 4, a derived table[0078]30 relates to ablockset28 in the following way. Table30 has blockingvariable columns30a, one for each blocking variable in the data group. The other columns form aset30b, which is the union of the non-blocking variables from the constituent blocks26 ofblockset28. When distinct blocks are from the same dataset, they may include overlapping variables. In this case, the derived table contains only one column for each overlapping variable.
• Table[0079]30 hasrows30cfor each of the blocking variable values specified in some block from the blockset. Specifically, if there are N blocking variables, the new table will have a row for every N-tuple30dof blocking variable values, [valBV₁, valBV₂, . . . , val_N], where val_Xis one of the values specified for variable X in one of the blockset's blocks.
• Table[0080]30 has two types of columns: blockingvariable columns30a, anddata columns30b. Blocking variable columns have cells which contain the appropriate blocking variable values. That is, for a row associated to [val₁, val₂, . . . , val_N], the value for blocking variable i will be val₁. The data columns have cells that can be referenced by a row identifier, [val₁, val₂, . . . , val_N], and a non-blocking variable from one of the blocks in the defining blockset, e.g. NBvar from block K.
• Consider the cell of the new table characterized by [val[0081]₁, val₂, . . . , val_N] and NBvar. If there is a block such that:
• 1. [val[0082]₁, val₂, . . . , val_N] is an allowed blocking variable combination, and
• 2. NBvar is an included variable,[0083]
• then this cell is filled in with the value from that block's dataset. Note that this value is the same if there are 2 such blocks in the blockset. If, for all such blocks, that cell does not exist, then this cell is empty in the new table. If {[val[0084]₁, val₂, . . . , val_N], Nbvar} is not an allowed combination for any block included in the dataset, then the cell is empty in the new table. Hence this algorithm produces an outer join of the multiple dataset tables referenced in the blockset's constituent blocks.
• Derivation Process[0085]
• Referring to FIGS. 5A and 5B, a[0086]derivation process56 generates a blockset derived table30 from a givenblockset28. This discussion assumes the dataset tables are stored in an SQL database, or some other container such that the data can be accessed via an SQL-like query language.
• [0087]Derivation process56 sequentially returns column values and table rows of a blockset derived table30. Each returned row is represented as a list of values. The first row is header row, listing column names.
• Initially,[0088]derivation process56 defines an order for the blocking variable set30a(step56a), either arbitrarily or by user response.Derivation process56 also defines an order for the blocks in the blockset.
• [0089]Derivation process56 generates a hashtable of query result sets (step56b). Most standard query engines allow a client to define a result set and retrieve the data row by row. This allows the client to make only one query per block, and then use the natural ordering of the returned data to progressively fill in the newly generated table. The result sets are indexed by a block id, where the id reflects the block ordering defined above.Derivation process56 therefore generates a hashtable which maps integers to data result sets from which data can be incrementally retrieved.
• [0090]Derivation process56 then connects to each of thedata sources18a(FIG. 1A) that provide the underlying datasets for the blocks in the blockset (step56c).Derivation process56 returns the first row (step56d), generating the row incrementally, starting with the blocking variable names in their specified order.Derivation process56 completes the row by going through the ordered blocks, appending each block's variable list.
• [0091]Derivation process56 traverses the result sets (loop bounded by56eand56n) in the same order as the rows that will be returned to the user. This way, no further requests need be made to the query engine. This ordering is created in the following way. For each dataset from which a block has been defined, generate a variable list V (FIG. 5B) which includes both blocking and non-blocking variables. Also generate the corresponding database table name D. Label the blocking variables BV1, BV2, . . . , and denote the union of values that BV1 takes on in all blocks with the set {A, B, C, . . . }, denote the union of values that BV2 takes on in all blocks with the set {X, Y, Z . . . .}, and so forth. Then for each block,derivation process56 submits a query of the type shown in FIG. 5B, during connection to the underlying datasets (step56c).
• Alternatively, if the number of allowed values for a blocking variable is so large that the resulting query is unmanageable, the above query can be run without imposing any restriction on the value of blocking variables. The resulting query would then return data for blocking variable values which[0092]derivation process56 would simply ignore. One benefit of this strategy is that, at this point in the process, several queries have been issued to query engines, and several connections are open. These connections are subsequently left open simultaneously, and data is retrieved incrementally, as needed. This enables the query process to run quickly and efficiently, more so than, for instance, standard queries against data cubes.
• [0093]Derivation process56 returns data for the blockset derived table30 one row at a time, creating one row for each distinct n-tuple (step56f).Derivation process56 proceeds from one n-tuple to the next in the same order that the SQL engine uses when it returns data via “ORDER BY BV1, BV2, . . . .” In particular,derivation process56 progresses through the blocking variable values alphabetically, as if the n-tuple was concatenated into one long string. For example, this can be done by first looping through the values of the nth-blocking variable, returning it to its starting point, incrementing the n-1st variable value, and so on. In this way, every row of the resulting new data table is returned.
• For each combination of allowed blocking variable values, i.e., for each n-tuple,[0094]derivation process56 goes through the blocks in the block set (loop bounded by56gand56k). For each block,derivation process56 retrieves a row of data from the corresponding result set, comparing the blocking variable values of that row to the current n-tuple. If it is a match,derivation process56 adds the retrieved non-blocking variable values to the data row being created for table30 (step56h).
• After all the blocks have been checked for a given n-tuple,[0095]derivation process56 returns the data row (step56m). Often, some of the data values will be empty. These may be represented by a “.” in the returned list. The resulting dataset can be further modified via standard techniques to be read into various programs; e.g., XML output, spreadsheet programs such as Excel or Quattro Pro, database programs such as Oracle, Access, and SQL Server, statistical programs such as Stata, SPSS, and SAS, and so on.
• User Interface[0096]
• Referring to FIG. 6, a[0097]login web page60 contains alogin section60a, a set oflinks60bto selected datasets, a set ofinstructions60con how to use the page, and a set ofquick links60dto other sections of the site. In this embodiment, the system assigns a unique client identifier44a(FIG. 1C) to each user22, which is then used to access preferences and previous saved data sets for that user. In general, the user need only be able to enter the system by supplying his or her login information, and be given access to the datasets and previously stored data carts.
• Browsing Area[0098]
• Referring to FIGS.[0099]7A-7E, amain browsing section62 contains alink bar62a, which may be identical to60dor contain different options.Section62 also contains a frame for a list of allavailable datasets62band abrowsing area62c. The datasets listed in62bmay be identical for all users, or they can vary depending on the user's identification and level of access privileges, stored for example in admin level44m(FIG. 1C). Thedatasets area62balso displays a Search option, which implements a search routine on all dataset titles, variable names, and descriptive metadata. If thesystem10 hosts only one type of data, such as data grouped by country and year, then this browse page can be accessed directly.
• If the[0100]system10 has more than one type of data, then an intermediate selection stage allows the user to choose which type of data she wants to view, according to the data's blocking variables. These data groups could include, for instance, country-year, firm-year, firm-quarter, congressional district, stock-day, and so on. Once a type of data is selected, the system examines the metadata for the data sets currently available and selects those data sets of the currently selected data group. The browsing page for that data group is then dynamically generated and displayed to the user, and only those data sets in the system of the selected group are shown inarea62bof the resulting page.
• FIG. 7B shows the[0101]main browsing section62 with thebrowsing area62creplaced by thedata cart summary62d, thedataset summary62e, and thedata selection area62f. This screen results when the user clicks on one of the datasets listed in62b. The data shown in thedata cart summary62dsummarizes the current blockset.Data cart summary62dincludes anicon63 that has an image of a physical shopping cart. Thedataset summary62eis constructed from dataset metadata stored inapplication database20. The cart summary contains descriptive information on the current data cart, described more fully below. The dataset summary contains summary information on a given dataset and a Quick Download option, in which the entire dataset is downloaded as a single blockset, using derivation process56 (FIG. 5A). The Add Data to Cart option adds the blockset consisting of the entire dataset to the current cart.
• The data selection area contains a number of tabs: one for each blocking variable and one for all other non-blocking variables. In the present example, the blocking variables are country and year, while the non-blocking variables are SYSTEM, YRSOFFICE, FINITTRM, etc. The system allows the user to choose one or more variables; in the example, this is done through checkboxes. It also allows for the immediate download of a subset via the Download Subset link, which downloads the currently selected blockset as defined by the choices made on the tabs. Various default options are available if the Download Subset link is pressed before all the tabs have been filled out; unused tabs may be assumed to be empty, for instance, or have all possible choices filled. Similarly, the Add Subset to Cart option adds the subset defined by the tabs to the current cart. Not all the variables in a dataset need be available to all users; dataset access, variable access, and data point access can all be limited via security clearance codes and data filtering. Each variable can also have one or more comments associated with it, stored in[0102]description44h(FIG. 1C). The user can access controls to create comments by clicking on the comment link next to any variable. These comments can be emailed to the system administrator and other users via standard mailing programs.
• FIG. 7C shows[0103]main browsing section62 with theyears tab62gselected. In the illustration, the user can select all years, select individual years one by one, or select years in groups decade by decade. Individual controls exist for each value the variable can adopt. Similarly, FIG. 7D shows the same view, but with thecountries tab62hselected. In the example, countries can be selected one at a time, or in groups by continent or international affiliations.
• Data Cart[0104]
• Referring to FIG. 8A, an example[0105]data cart screen66 results from the user's clicking on the Add Data to Cart button or the Add Subset to Cart button, shown in FIG. 7A. Thecart summary area66adisplays summary information about the current cart, while the list ofvariables66bcontains a list of the variables currently included in the data cart50 (FIG. 2), and any relevant metadata information. Indata cart screen66, thedataset listing62band datacart summary information62dremain, as inbrowsing area62. Additionally,data cart screen66 allows a user to view and generate thecodebook66cassociated with the data cart50.Codebook66cis a custom-generated list of the metadata associated with the variables in the current blockset. Referring to FIG. 9, asample codebook window68 includes themetadata68afor a sample variable.
• Thus the process of selecting data in[0106]system10 allows users to place variables in their data cart50, just as online shopping providers use a shopping cart for goods and services. The process of selecting data for data cart50 is analogous: users place variables in their data cart. The cart analogy makes system navigation easy and intuitive; the users need only select the variables that they want and put them into a cart. They can combine variables from more than one dataset as long as all datasets belong to the same data group. When all desired variables have been added, the user can “check out” the cart by downloading the data, as described below. This construction also allows for asynchronous data selection; users can build their custom-made data sets a little at a time, as opposed to systems in which the variable selection must be made all at once. Data carts can be created and modified quickly due to their underlying blockset construction, whereby variables can be added and subtracted from blocksets via the manipulation of their metadata only, allowing the browsing process to occur without noticeable system delays.
• As the user browses the website, she can use web forms to define data Blocks by choosing a dataset and specifying year and country subsets. The most recent block definition is stored with the user's session information. This block is used when new datasets are browsed; their most recent year and country selections are pre-filled into the forms associated to each dataset viewed.[0107]
• If, while using a dataset Block definition form, a user chooses the “Add to Cart” feature, the currently defined Block is added to the list of blocks making up the current data cart blockset for that user. At any time, a user can view a web page which shows a list of the blocks contained in their current Datacart. This page is generated by iterating through the distinct Blocks in the Block list which defines the users datacart object. From this page, the user can choose to “Remove” individual Blocks from their cart. When the remove operation is requested, the system shortens the list of blocks in the user's datacart by one item, and the specified block is no longer referenced by the datacart object.[0108]
• A user session can continue, with the user defining, adding, and removing Blocks from their dataset multiple times. If and when they decide they want to “Save” this cart, the system prompts for information about the cart. In particular, referring to FIG. 1C, the user provides a[0109]title48, adescription48c, and apublic flag48ffor the cart. The entered title and description are combined with the privacy flag, the user's ID44a, the current date, and a unique ID (blockset key48a). This information is saved intoapplication database20. A description of each Block in the datacart's list of Blocks is saved as a separate row in block table46. Rows in this table contain ablockset FK46ccolumn which references back to this cart's ID. In this way, blocks for a specified cart are uniquely identified.
• Data Cart Information[0110]
• Referring again to FIG. 8A, the user has the option of editing her data cart either by using the[0111]edit button66dor theSave Cart feature66e. Each of these leads to a datacart editing screen70, shown in FIG. 10. The data cartinformation form70acan contain any of a number of fields relating to the summary cart information. The cart may also be saved as public or private. If the former, then any other registered user may view that data cart; if private, then only the user herself may view the cart in the Archive, described below. These choices need not be dichotomous; intermediate levels of access can be specified as well, depending on the particular organization's needs. For instance, only registered users with a certain clearance level or above may have access to certain data and saved carts.
• After variables have been added to the data cart, the user can return to browsing the data sets, as illustrated in FIG. 7E. Any other dataset can be selected, and its variables can be added to the variables currently in the data cart. In the illustrated embodiment, the system displays the choices on the blocking variables made on the previous blockset added to the cart. In the illustration, the previous countries and years are both automatically selected and highlighted, the countries tab being shown as[0112]62h, with highlighted previous choices as72i.
• Referring to FIG. 8B, if the user adds more variables to their data cart, then the list of variables in the[0113]cart display66fis expanded to include the variables from all data sets added to the cart. Items can be removed from the cart either one at a time via66g, or in their entirety via66h. Summary information from the cart is available via66i. As above, the entire blockset can be viewed withcontrol66jor “checked out”—downloaded—withcontrol66k.
• Archive[0114]
• Referring now to FIGS. 11A and 11B, the user accesses[0115]archive section72 through thestandard toolbar60d. One area ofarchive section72 shows savedcarts72a, as well ascarts72bfrom other users who have placed public carts on the system. The user can delete her own carts. When another user's name is selected, then their publicly saved carts appear. Clicking on any saved cart brings that cart's information into theviewing area72c, as illustrated in FIG11B. This shows the variables in the savedcart72d, which can then be manipulated as any other blockset.
• The[0116]archive section72 allows for collaboration by geographically dispersed users. Datasets can be saved, edited, and then saved again online, by manipulating the metadata of the saved blocksets. This makes dataset storage inexpensive from the viewpoint of system resources, and it makes the saving and retrieval of data carts quick and efficient. As with the data cart, the archive area also allows the user to select variables asynchronously. Not only can data be added to a cart a little at a time over a single login session, but saved carts add the possibility of stretching the dataset creation process across multiple sessions without having to rebuild the dataset from scratch every time.
• Dataset Upload[0117]
• Referring to FIG. 12A and FIG. 12H, an[0118]administrator screen74 is accessible to users with administrative privileges. The left-hand pane74bin FIG. 12A lists administrative options, including List All Users, and Adddatasets74n. When the user selects thecontrol74nto add a dataset, the system initiates a wizard interface as illustrated in the right-hand pane74a. In the example, the databases to be loaded into the system are first located within a single directory on the server; hence only the database name need be entered. In alternative embodiments, a fully qualified directory path or URL could be entered, requiring only that the server have access privileges to the specified location. The database name is a value to be stored in database name field36dof dataset table36 (FIG. 1C).
• Upon entering this information, the user is taken to the[0119]screen74 illustrated in FIG. 12B, where the user selectsmetadata codebook74canddataset74d.Dataset74dis a control that specifies a data source table84 (FIG. 12H) of data that will be the basis for adataset24. The table is identified within its database by a datasource table name84d.Codebook74cis a control that allows the user to specify a codebook table82, whichsystem10 uses as a source of metadata to describe thedataset24 to upload. Codebook table82 is identified within its database by codebooktable name82f. If the dataset uploads successfully,system10 stores this value in the field codebook name36gof the corresponding row in dataset table36 (FIG. 12H).
• In general, a codebook table[0120]82 stores information used to identify adataset24, such as the dataset name, the set of blocking variables, and the names of the non-blocking variables. Other useful metadata can be added as well, such as author, variable descriptions, and coding rules.
• Each row of the codebook table[0121]82 represents information applicable either to an entire dataset or to a variable within a dataset. When a row includes the keyword “dataset” incode field82, the row represents adataset24. Otherwise, the row represents a variable—in particular,code field82 gives the name of a non-blockingvariable field84bin data source table84.Title82cstores a name for the corresponding variable.Description82dstoresphrases describing dataset24 in detail. Author FK82eis a foreign key referencing author table40, which allows a dataset or variable to be associated with a particular author.
• Referring to FIG. 12C and FIG. 1C, after entering the dataset and codebook locators, the user is taken to screen[0122]74, which lists the non-blockingvariable names74eas well as the actual values of the blockingvariables74f. Upon user confirmation, the data can be added to the system. Each non-blocking variable has a corresponding row in codebook table82, which serves as the basis for a new corresponding row in variable table38. In particular,title82cmaps to title38c,description82dtodescription38e, and code82bto name38g.
• The row in codebook table[0123]82 with “dataset” in code field82bcorresponds to a new row in dataset table36. In this case,description82dmaps to description36f.
• Other administrative features can be added to the system as well, including facilities for maintaining user accounts, assigning privileges, and editing metadata. An illustration of the latter is provided in FIG. 12D, where the administrator is presented with a[0124]form74gthrough which she can modify a dataset's author and category information. The author editing screen is illustrated in FIG. 12E, which displays thecurrent author information74h, the list of potential authors currently in thesystem74i, and aform74jfor adding a new author. Similarly, FIG. 12F offers the possibility of adding or amending category information. The current set of categories is provided in74k, possible additional categories already in the system are provided in74m, while new categories can be added via74p.
• Referring to FIG. 12G, an upload process[0125]80 guides a user through a process of adding adataset24 to the system, verifying that the necessary metadata is in place. Upload process80 receives adata group34, either explicitly as a passed value or reference, or implicitly by a default value (step80a). Thedata group34 has a set of blocking variables.
• Upload process[0126]80 presents a user interface that prompts a user for a database name (step80b). Upload process80 then receives user input specifying database name, which upload process80 stores (step80c). Upload process80 compares the database name to its current set of databases18 (FIG. 1A) and determines whether the database name specifies adatabase18 thatsystem10 can connect to, for example using an ODBC connection (step80d). If so (result80e), upload process80 connects to the database and retrieves a list of tables in the database to present to the user (step80g). Otherwise, if the database is not available (result80f), the process prompts again for database name (step80b).
• The user can choose a data source table[0127]84 from the list of tables, as well as a codebook table82 (FIG. 12H). Upon receiving input specifying the user's choice (80h), upload process80 prompts the user to confirm blocking and non-blocking variables in the data source table (step80i). Upload process80 tests whether the user input confirms that the blocking and non-blocking variables are identified correctly (step80j). With confirmation (result80k), upload process80 retrieves rows from codebook table82 that describe the dataset and variables (step80p). If the user does not provide confirmation (result80m), upload process80 returns failure (step80n) and terminates without adding adataset24 to the system.
• Upload process[0128]80 verifies that the rows in codebook table82 correspond to the dataset and variables of data source table84 (step80q). If any row is missing (result80r), upload process80 returns failure (step80n) and terminates without adding adataset24 to the system. Otherwise (result80s), upload process80 prompts the user and receives input to confirm the metadata for theprospective dataset24 itself, as opposed to the metadata for the variables (step80t). Such metadata for the dataset includes its storage name, its name in presentation to users (i.e., a caption), a textual description, and its location.
• If the dataset metadata is confirmed (result[0129]80u), upload process80 loops to confirm each of the variables (loop bounded by80wand80ab). Otherwise, if the dataset metadata is not confirmed (result80v), the process returns failure (step80n) and terminates without adding adataset24 to the system.
• Upload process[0130]80 iterates over each variable to prompt the user with the associated metadata (step80x), for example the storage name, the title, a textual description, associated keywords, and units. Upload process80 tests the user's response (step80y). If the user rejects any variable's metadata (result80aa), the process returns failure (step80n) and terminates without adding adataset24 to the system. Otherwise (result80z), upload process80 commits thedataset24 toapplication database20 and adds acorresponding database18 as the provider of the underlying data.
• Codebook Download[0131]
• When a user asks to download the codebook information about a specified dataset or saved datacart, the system iterates through each block of the datacart. For each block, it retrieves all codebook records which reference either the dataset containing that block, or a variable within that block's definition. This codebook information is stored in memory and formatted for display in a web page, or for printing. When a secondary table row is referenced, e.g. when an author is specified, the author table is queried, the information is retrieved and then formatted.[0132]
• Referring to FIG. 13 and FIG. 1C, a codebook process[0133]86 receives a request for a codebook for a table derived from a blockset (step86a). Codebook process86 retrieves metadata for blockset fromapplication database20, forexample title48eanddescription48cfrom blockset table48 (step86b). Codebook process86 outputs the metadata for blockset (step86c), then loops over the variables of the derived table (loop bounded by86dand86g), as encoded in the rows in variable table38. For each such variable, codebook process86 retrieves metadata from the application database (step86e). Such metadata includes a variable title38c, adescription38e, and the title36bof the associated dataset in dataset table36.
• Alternative Embodiments[0134]
• A number of embodiments of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. For example,[0135]databases18 can be located on other machines thanserver12, and can be connected via a database server. Thus theserver12 can include multiple physical machines distributed across a network.
• In the described embodiment, each[0136]block26 has oneblockset28. This provides an administrative advantage, in that edits to the block of afirst blockset28 cannot affect other blocksets, since blocks are not shared. In alternative embodiments, however, blocks could be shared by blocksets.
• In the described embodiment, codebook table[0137]82 includes basic text information like descriptions and comments. In alternative embodiments, codebook table82 could also reference rows in other tables ofapplication database20, such as citation records or categories.
• Accordingly, other embodiments are within the scope of the following claims.[0138]

Claims (25)

What is claimed is:

1. A computer-based method of representing a subset of a dataset table of rows and columns, the method comprising:

selecting a set of blocking variables corresponding to blocking columns of the dataset table such that, for each row in the dataset table, a tuple of values for the blocking columns uniquely identifies the row within the dataset table;

selecting a set of non-blocking variables corresponding to columns of the subset, the set of non-blocking variables disjoint from the set of blocking variables; and

creating a block information structure that includes the set of non-blocking variables and that includes, for each blocking variable in the set of blocking variables, a set of values.

2. The method ofclaim 1, wherein for each row in the subset having a tuple of values for the blocking columns, the values of the tuple are included in the corresponding sets of values for the blocking variables.

3. The method ofclaim 1, wherein the subset of the dataset table includes the entire dataset table.

4. The method ofclaim 1, wherein a sum of the number of elements in the set of non-blocking columns and in the sets of values for the blocking columns is linearly proportionate to an upper bound on binary storage requirements of the block information structure.

5. The method ofclaim 1, further comprising:

storing the block information structure on a machine-readable medium.

6. A computer-based method of accessing information in heterogeneous databases, the method comprising:

presenting a graphical user interface with controls representing a data cart and a plurality of datasets;

receiving user input that selects a dataset to add to the data cart from the plurality of datasets;

generating a block information structure specifying the dataset; and

adding the block information structure to the data cart.

7. The method ofclaim 6, wherein specifying the dataset includes incorporating into the block information structure a set of non-blocking variables, a set of blocking variables, and for each such blocking variable, a set of values.

8. The method ofclaim 7, wherein the dataset includes a plurality of rows, each identified by a corresponding tuple of values from the sets of values for the blocking variables.

9. The method ofclaim 7, further comprising:

for a blocking variable in the set of blocking variables, presenting enumeration controls in the graphical user interface, each corresponding to an existing value in the dataset for the blocking variable.

10. The method ofclaim 6, further comprising:

collecting user input specifying a subset of the dataset, and representing the subset in corresponding block information structure.

11. The method ofclaim 6, further comprising:

saving the data cart to a persistent storage medium.

12. The method ofclaim 6, further comprising:

adding a second block information structure to the data cart, in response to user input.

13. The method ofclaim 6, wherein the controls representing a data cart include a symbol of a shopping cart.

14. A computer-based method of retrieving information represented by a blockset, the method comprising:

for each of a plurality of blocks in the blockset in which each block specifies a subset of a dataset stored in a database, connecting to the corresponding database, wherein the blockset has a plurality of blocking variables, and the blocks each include a set of non-blocking variables, and specifying includes having a set of values for each blocking variable in the plurality of blocking variables;

adding to a derived table a blocking column for each blocking variable in the set of blocking variables;

adding to the derived table a non-blocking column for each element in a union of the non-blocking variables in the plurality of blocks;

adding a row to the derived table including a cell for each column in the derived table, the row uniquely identified by a tuple of values from the sets of values for the blocking variables; and

populating a cell of a non-blocking column in the row, using a value retrieved from the database corresponding to the block, wherein the block contains the non-blocking variable corresponding to the non-blocking column.

15. The method ofclaim 14, further comprising:

adding a row for each tuple of values from the sets of values for the blocking variables, wherein the tuple occurs in at least one dataset corresponding to a block in the plurality of blocks.

16. The method ofclaim 14, wherein connecting includes using each block as a basis for a database query that specifies the corresponding subset.

17. The method ofclaim 16, wherein the database query uses Structured Query Language.

18. A computer-based method of representing a table derived from a blockset, the method comprising:

outputting blockset metadata describing the blockset, including fields for a blockset title and a blockset description;

for a column in the table, outputting column metadata describing a variable associated with the column, wherein the variable is associated with an underlying dataset that provides data to the table in the blockset, and the column metadata includes fields for a title of the variable and for a title of the underlying dataset.

19. A computer-based method of collecting metadata for a dataset, the method comprising:

prompting a user to provide a database name;

confirming that the database name represents a database;

displaying a list of tables in the database to the user;

receiving user input specifying a table in the list of tables;

prompting the user to confirm that a list of blocking variables and a list of non-blocking variables are correct for the database;

prompting the user to confirm metadata for the dataset and for the list of non-blocking variables;

if the user confirms the list of blocking variables, the list of non-blocking variables, and the metadata, adding a dataset corresponding to the table to a collection of datasets.

20. The method ofclaim 19, wherein the metadata includes a title for the dataset.

21. The method ofclaim 19, wherein the metadata includes a description for the dataset.

22. The method ofclaim 19, wherein the metadata includes a title for a non-blocking variable in the list of non-blocking variables.

23. The method ofclaim 19, wherein the metadata includes a description for a non-blocking variable in the list of non-blocking variables.

24. An article comprising a machine-readable storage medium that stores executable instructions to represent a subset of a dataset table of rows and columns, the instructions causing a machine to:

select a set of blocking variables corresponding to blocking columns of the dataset table such that, for each row in the dataset table, a tuple of values for the blocking columns uniquely identifies the row within the dataset table;

select a set of non-blocking variables corresponding to columns of the subset, the set of non-blocking variables disjoint from the set of blocking variables; and

create a block information structure that includes the set of non-blocking variables and that includes, for each blocking variable in the set of blocking variables, a set of values.

25. An article comprising a machine-readable storage medium that stores executable instructions to represent a subset of a dataset table of rows and columns, the instructions causing a machine to:

present a graphical user interface with controls representing a data cart and a plurality of datasets;

receive user input that selects a dataset to add to the data cart from the plurality of datasets;

generate a block information structure specifying the dataset; and

add the block information structure to the data cart.

Images