PROCESSING ENGINE
The present invention relates to the processing and egorisation of large volumes of data and, more particularly but not exclusively to financial reporting data.
BACKGROUND
Computers are known for their ability to process large volumes of data. However, in the case where the data, or at least parts of it, is unstructured there can be difficulties in the computer recognising the information contained in the data and knowing how to handle it.
There are many instances in industry where unstructured data presents a processing problem to data processing devices. For example accounting information in the form of trial balances from general ledger systems will present to a data processing system in an unstructured way and typically a different way depending on the general ledger system from which the data is derived.
The technical problem to be addressed is that of how to take a databit stream coded according to a first protocol and representing unstructured data and apply logical processing to that databit stream so as to produce tocol and representing structured data. The structured a can then have further processing performed on it so as achieve outcomes that could not be achieved either at all or in a reasonable time frame but for the preceding logical processing step.
It is an object of the present invention to address or at least ameliorate some of the above disadvantages.
Notes
1. The term "comprising" (and grammatical variations thereof) is used in this specification in the inclusive sense of "having" or "including", and not in the exclusive sense of "consisting only of".
2. The above discussion of the prior art in the .
Background of the invention, is not an admission that any information discussed therein is citable prior art or part of the common general knowledge of persons skilled in the art in any country. BRIEF DESCRIPTION OF INVENTION
In a broad form the processing engine of a preferred embodiment of the present invention when applied to a financial database automatically "tags" (also commonly referred to as "maps" or "allocates") information from an ined function codes, groups or fields; thus identifying populating .new and usable ways of filtering and regating the financial database information. The biggest advantages of this are elimination of time for manual creation; and also reliable .and consistent treatment of these new populations thus significantly reducing errors.
' Accordingly, in a further broad form of the present invention there is provided a processing engine for processing unstructured data .located in an unstructured data table; said data comprising quantity data associated with an unstructured descriptor element; said engine comparing each said unstructured descriptor element against a master list of mappable data descriptors to create a single data description per entry; in the event of a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with the matching mappable data descriptor and copied into a partly structured data table.
In a further broad form of the present invention there is provided a processing engine which receives a databit stream coded according to a first protocol and representing unstructured data; said processing engine applying logical ;abit stream coded according to a second protocol and >resenting structured data.
In a further broad form of the present invention there is provided a method for converting a databit stream coded according- to a first protocol and representing unstructured ■data via a step of logical processing to that databit stream so as to produce a second databit stream coded according to a second protocol and representing structured data.
In yet a further broad form of the present invention there is provided a processing engine which automatically "tags" (also commonly referred to as "maps" or "allocates") information' from an already existing financial database with, newly created and defined function codes, groups or fields; thus identifying and populating new and usable ways of filtering and aggregating the financial database information.
Preferably said partly structured data table is reviewed manually.
Preferably those unstructured descriptor elements for which no match is available are manually reviewed and a manual :ch made so as to produce a fully structured data table >m said partially structured data table.
Preferably those unstructured descriptor elements for which no match is available comprise less than 25 per cent of said unstructured data.
Preferably said unstructured data comprises accounting data. -
Preferably the structured data table can be filtered and aggregated by all data fields, including newly created data descriptions, thereby to provide new ways of viewing, summarising and disseminating data.
Preferably advantages of employing the processing engine are elimination of time for manual creation; and also reliable and consistent treatment of these new data descriptions thus significantly reducing errors.
In yet a further broad form of the invention there is provided a processing engine for processing unstructured data located in an unstructured data' table in a computer memory; said data comprising quantity data associated with an unstructured descriptor element; said engine including a ment against a master list of mappable data descriptors create a single data description1 per entry; in the event a match signal from said comparator indicating a match between an unstructured descriptor element and a mappable data descriptor in the mappable data descriptors list then quantity data associated with said unstructured data element is associated with, the matching mappable data descriptor and copied into a partly structured data table in said computer memory.
'
Preferably said database and said comparator are implemented via a microprocessor in communication with a computer memory. BRIEF DESCRIPTION OF DRAWINGS
Embodiments of the present invention will now be described with reference to the accompanying drawings wherein:
Figure 1, . is a block diagram of dataflow through a processing engine in accordance with a first preferred embodiment of the present invention,
Figure 2, is a logic flow diagram of processing steps applied to data in accordance with the first embodiment,
Figure 3, illustrates data mapping for a particular instance of data, Figure 4 is an example of a fully structured data de,
Fig 5 illustrates a further example of the tagging process where translation is to a common descriptor base - in this case an XBRL common descriptor set,
Fig 6 is a simplified explanation in table form of an embodiment of the technology process showing how the Master Chart Of Accounts tag may be determined.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
With reference to Figure 1 there is illustrated a processing engine 10 in accordance with a first preferred embodiment of the present invention. In this instance the processing engine 10 comprises a translator 11 which receives input from comparator 12. The comparator 12 receives and compares data in unstructured data table 13 and mappable data descriptor table 14. The output of translator 11 is input to partly structured table 15. A manual processing operation is then applied to table 15, the output of which results in a fully structured data table 16.
With reference to Figure 2 a logic flow diagram is provided for processing steps applied to data passing tance, the data relates to trial balance data. In this tance the processing engine is termed the itoallocator" in Figure 2.
FRS AutoAllocator - outline
Using the FRS AutoAllocator (part of the FRS Process), we can define the process as:
' Importing Trial Balances from any General Ledger system and the FRS System (part of the FRS Process) automatically knows what Master Chart Of Account Allocation (based on data input from the International Financial Reporting Standards) to give it.
For example it would recognise:
GL Account Name Master Chart
Till floats as Cash on hand
estpac account as Cash at bank
Figure 3 illustrates data mapping in the instance where data is trial balance data and comprises a listing of unstructured descriptor elements 17 and a corresponding listing of mappable data descriptors 18. So, for example, unstructured descriptor element 17A comprising the •descriptor "till floats" maps to 'a matched mappable data descriptor 18a being descriptor "cash on hand."
In summary the processing engine automatically "tags" (also commonly referred to as "maps" or "allocates") information ated and defined function codes, groups or fields; thus ntifying and populating new and usable ways of filtering aggregating the financial database information. The biggest advantages of this are elimination of time for manual creation; and also reliable and consistent treatment of these new populations thus significan ly reducing errors.
Explanation of tagging process
The "tagging" (also commonly referred to as "mapping" or "allocating") process uses elements of existing financial database fields (such as the general ledger code, account name / description or balance) to predict a new field (such as a Master Chart of Accounts category or XBRL Value) using, amongst other describable processes, a matrix and array linear pattern. The technology also contains an inbuilt validation process, largely centred around the GL Code (with ranges input to match the current GL,' which varies from system to system) and the GL Balance.
The tagging process is instantaneous - once you import the financial database (e.g. the trial balance), the new function codes (e.g. Master' Chart of Accounts) appears immediately. 5 illustrates a further example of the tagging process re translation is to a common descriptor base - in this e an XBRL common descriptor set. Fig 6 is a simplified explanation of the technology process as to how it determines the Master Chart Of Accounts tag.
In Use
With reference to Figure 4 a fully structured data table 16 is illustrated with reference to a particular example. In this instance the unstructured descriptor "Westpac account" is associated with quantity .ql . It is mapped by way of the table of Figure 3 to descriptor "cash at bank" associated with quantity ql. Similarly, the descriptor "trade debtors," is initially associated with quantity q2 and is mapped to be associated with descriptor "trade receivables."
In the case of the descriptor "special advance" there is no mapping available from the table of Figure 3 so, initially, no mapping takes place in the creation of the partially structured data table. The manual data entry step is then performed whereby the descriptor "special advance" is mapped to "other receivables" and remains associated with quantity qn.
Having produced the fully structured data table the it in an automated fashion, it will be appreciated that more comprehensive the mappable data descriptor table the smaller the amount of manual allocation will be as a proportion of total amount of data processed thereby resulting in the potential for relatively rapid processing of large volumes of data presenting as unstructured data from disparate sources.
The above describes only some embodiments of the present invention and modifications, obvious to those skilled in the art, can be made thereto without departing from the scope of the present invention.