Method for the use of digital cameras and cameraphones.
The present invention is related to a method for the use of digital cameras and cameraphones equipped with intelligence for the detection and recognition of a symbol displayed in an arbitrary environment on any media, such as newspapers, magazines, brochures, posters, billboards, LCD- and other electronic screens and presentation means.
By my Swedish patents Nos. 517 295 and 519 405 methods, means and applications for the use of advanced digital cameras and mobile phones equipped with a digital camera, in the following denoted cameraphones, are disclosed, the overall vision being to transform these now widely spread devices from only "seeing" to "seeing, interpreting and acting". To achieve this, the advanced digital cameras and cameraphones will have to apply various levels of artificial or machine intelligence (Al) for the purpose of symbol recognition, and will be referred to as intelligent digital cameras. The Al field is constantly developing and new methods are being introduced, based on neural networks and algorithms of ever increasing complexity. There is, however, a considerable gap between the front end research in the Al and machine intelligence fields and practical reality, mainly caused by the fact that robust methods for symbol recognition and interpretation in a general environment are still lacking.
The object of the invention is to provide a method according to the pre-characterizing portion of claim 1, wherein the above problems are solved.
This object is achieved by means of a method according to the characterizing portion of claim 1.
The subclaims disclose preferable embodiments of the invention.
The method according to the invention shows a number of approaches, by which the above mentioned problem can be overcome in a very efficient way, greatly facilitating the rapid introduction of practically very important and robust every day life uses of intelligent digital cameras and cameraphones.
To elucidate this, two use cases will be described in a hands-on way.
The Use case # 1 elucidates the application of cameraphones for fully mobile, quick and effortless booking & reservation of tickets and seats to any type of events, such as cinemas, theatres, rock concerts, sport events, last minute flights, etc.
The Use case # 2 similarly elucidates the application of cameraphones and digital cameras equipped with communication means to computers (including communication via, e.g., IR, Bluetooth and cables) for ordering of goods and services within the mail order and distance selling fields.
As these applications have several aspects in common, the Use case # 1 will be described in more detail and implied reference will be made to this information when describing the Use case # 2. When in the following reference is made to the use of a cameraphone, it will include the use of any digital camera equipped with communication means to a computer or server with a data base.
Use case # 1
An excellent high volume application for intelligent cameraphones is for booking and reserving tickets for cinemas, operas, theaters, rock concerts, sport events, last minute flights, etc. The common denominator for these types of events is that only a few parameters are required to specify on the one side the offer by the event organizer, on the other side the requirements of the user, namely name and date of the event, number of tickets required and placement requested.
In the following, the steps involved in the particular use case of reserving seats for a cinema from advertisements in newspapers will be described. The other types of events have very similar use cases. For the sake of concreteness, it will be assumed that a specialised company, InMoDo, has developed this application and is using its code carrying logotype and a special booster coding technology, with two codes which are easily recognized by the digital camera, to make the application very versatile both for the end user and the event organizer.
Although it is not in any way a necessary precondition that one or several codes be used to target the camera to the right server based application as explained in the following, the booking and ordering applications can easier achieve a high degree of automation, efficiency and speed if the advertisement contains some unique digital codes, such as the ones introduced below.
We will therefore assume that two codes are actually displayed on the advertisement, poster, or other visually recognizable medium communicating an offer from an event organizer or a supplier to the general public. These codes, in the following referred to as the targeting codes, are markers resembling standard bar codes but for technical reasons are preferably designed as framed planar digital bit structures of, e.g., 8 x 8 or more bi-tonal (typically black and white) areas or dots with, preferably, inherent error correcting and redundancy characteristics, which are not the subject of this patent application. The targeting codes may involve check sums and various error correction techniques, such as forward error and Reed Solomon error correction, use edge linking methods to give lighting variation insensitivity and be designed in order to make possible correction for perspective distortion and allow homography mapping. An 8x8 or 10x10 bit quadrilateral structure will give enough redundancy to also correct for rotational ambiguity. The quadrilateral contours may be found by edge based methods, which have robust detection properties.
The targeting codes in the following examples are, respectively, the
• i - code, which is also a logotype which instantly tells the cameraphone user that the offer he sees can be read and recognized by his unit, while its coded information immediately targets the camera to the realm or main set of intelligent digital camera applications involving symbol recognition for preferably, but not exclusively, commercial purposes, and the • a - code, which is the application and offer specific code, directing the camera to a particular application or subset in the realm of intelligent applications
In intelligent digital camera applications, the i - and a - codes can be said to have a similar function as the www - prefix and the home page address have, respectively, on the Internet. It can be shown that by using two codes, a booster effect is achieved in the sense that false signals, and in particular false positive signals which might target the user to a wrong application, can be almost completely avoided.
For the purpose of this description it may be assumed, that event organizers and suppliers of goods and services are licensed to use the i - logotype and code and will receive from time to time over the Internet from the operator of the server which carries the service, as explained in the following, the a - code by which any special offer to the public is tagged in context with advertising the offer.
The significance of these aspects also lies in the fact that all ordering applications, such as ordering of goods or services or booking of tickets, have common characteristics, distinguishing them from other intelligent digital camera applications, such as reading and translating an arbitrary text. It is thus useful if visual messages from suppliers and event organizers carry both a logotype and codes, which enables the end-user to know that the offer can be reacted upon and automatically utilized by just a snapshot of his intelligent cameraphone, at any time and wherever the user is located. This technical advantage has also obvious commercial advantages for all parties in the supply chain, but is not imposing an artificial constraint because, for any access mode which a supplier or event organizer wants to offer the public, be it the telephone, the Internet, e-mail or ordinary mail, it is anyhow necessary for the supplier to identify himself and the offer by suitable information including, e.g., home page address, e-mail address, or postal address. Of all these possible access modes, accessing the offer automatically via the i - and a - codes as recognized by means of a snapshot by a digital camera, is the most effortless one from the end user's point of view, freeing the end user from waiting time in telephones, or starting up a computer and entering home page addresses, or going to a mail box, etc.
With the above background and definitions, the consecutive steps of the particular booking application will be described below.
To illustrate the versatility and unique advantages of the booking application, we may consider a case when the end user sits on the subway on his way to work, and reads a newspaper. In the entertainment section, he finds an advertisement for a new movie. He decides that he immediately wants to reserve two tickets on the opening night. Looking at the advertisement he recognizes the i - logotype and the i - and a - codes in the corner of the ad. He catches his cameraphone, selects the intelligent mode (i.e., the before mentioned symbol recognition mode which was preinstalled on his cameraphone or which he has downloaded at a reasonable cost from InMoDo's home page via the JAVA option of his unit), and takes a snapshot of the advertisement. The advertisement may be illustrated by Figure 1A exhibiting the printed targeting codes at the lower left end of the ad.
Once the snapshot is taken and sent by pressing a send key or by another suitable instruction, the subsequent dialogue between the user, the application carrying server and the event organizer is illustrated by Figures 2 and 3.
The captured image is now processed fully automatically by the software of the cameraphone, using one of several available computer vision techniques to first find the i - code. For this purpose correlation techniques may be used, but the code envisaged herein allows more robust identification techniques to be applied.
Next to the i - code is the application and offer specific a - code (resembling the bar code identifying a product in a shop), which the program uses to automatically call and connect to the applicable service on the server via the mobile phone network, as illustrated by the left (bottom) part of Figure 3, and to detect that it is an advertisement for a specific movie at a given cinema and location.
Using the information contained in the a - code, the server then establishes contact with the event organizer's internet service, as illustrated by the right (upper) part of Figure 3.
Immediately thereafter, the event organizer's business system returns via the server a message to the cameraphone, as illustrated by Figure 2, and the cameraphone now displays an adapted Internet web page, on which the user can select date, number of tickets and number of seats.
It is obviously allowed within the scope of the invention that the server is located at the event organizer's or suppliers premises and/or is part of his business critical system, which would allow communication via, e.g., a LAN.
After having selected everything according to his preferences, the end user submits the information to the server. The server processes it, looking for available seats for the particular cinema and date, performs a preliminary reservation and then within seconds sends a confirmation page back to the end user with the preliminary reservation and statement of ticket prices, as further illustrated by Figure 2. (For the sake of simplicity, the ticket price information has been left out in Fig. 2).
The cameraphone user accepts the reservation by submitting that page and the server books the tickets and finally sends a confirmation e-mail to the end user, with all necessary information about the booking, as shown at the bottom of Figure 2.
From the above description and Figure 2 it follows that once the preferably quadrilateral structures containing the targeting codes in Figure 1 have been recognized and their interiors have been sampled to read the coded information, which normally will require only a fraction of a minute, the Internet link between the application server and the event organizer's or supplier's data base and business critical system, represented by the right (upper) part of the Figure 3, is utilized to match the end-user's request to the event organizer's or supplier's up to date information regarding ticket availability or stock balance, current prices , etc.
To make it possible to utilize the event organizer's or supplier's standard Internet interface, the server based application will automatically transform and adapt the format of the information from the cameraphone into a protocol, which is readily acceptable to the supplier's Internet link.
Thus, the booking or orders transmitted on the Internet link between the application server and the supplier's data base and business system will have a structure identical to that of booking orders entered directly via Internet, which greatly facilitates the introduction of image based services.
Conversely, information such as order confirmation returned to the cameraphone from the supplier, will be transformed by the server to a format suitable for display on the cameraphone and easily recognizable to the user, making the dialogue between user and supplier efficient and optimally adapted to each party's needs.
The system architecture and the consecutive process steps required to render the booking application are described in more detail in the following.
Process steps
Prefiltration
Basic image operations are performed in the cameraphone on the image taken, in order to reduce noise and improve contrast. i - code detection and interpretation
The pre-filtered image is then operated upon for the detection of the i - code by one of several available marker identification techniques, such as scale-invariant correlation computer vision techniques or edge based techniques for the detection of quadrilateral structures, the interior of which is then sampled for its digital content, assuring to a high degree that the image is indeed taken of an i - mode enabled object, and it also enables the software to know more precisely where to look in the next step for the a - code. a - code detection and interpretation
The pre-filtered image and the location information from the previous step is now used to find the a - code, which contains the specific information about which type of application it is and which particular event is involved, etc. The application specific symbol is recognized from a set of known symbols, using a recognition technique similar to the one in the previous step.
Web pages
With the information retrieved from the above steps, the application now enters into a web application mode, in which the user in addition to his booking also can choose to select more specific information about the event. With the currently available technology, these pages can be developed in Java Servlet pages with output in a format suitable for cameraphones (WAP or similar), and the access is done via GPRS.
Server requirements
Depending on the technological level of the cameraphone, different amounts of computation can be performed on it. For low end cameraphones (below denoted Type 1 telephones, as further explained in the following by reference to Figure 4) a large part of the computations will have to be performed on external servers, thereby increasing the requirement for bandwidth. More advanced cameraphones (Types 2- 3), will require less bandwidth and be less server dependent.
Type 1 cameraphones:
The cameraphone only captures and prefilters the image, whereupon it sends it to the server for further processing.
Type 2 cameraphones:
The cameraphone uses the prefiltered image to find by itself the probable location of the targeting codes, whereupon it sends the part of the image containing these symbols to the server.
Type 3 cameraphones:
The cameraphone uses itself the image part mentioned under Type 2 and recognizes the targeting codes and then sends the information contained in the a - code to the server.
Type 4 cameraphones:
In the booking application, the Type 4 (i.e. most advanced) cameraphone will interact with the system in the same way as a Type 3 unit, since the booking must be performed on a server.
Network requirements
From the above and the following it is evident that the network capacity utilized in the application is largely determined by the capability of the particular cameraphone used. According to the invention, the cameraphone will itself identify its model and capability category to the server, allowing the process to be conducted accordingly and fully automatically, with no requirement on the user to make any special input characterizing the capability of his unit. On the average, with low-end cameraphones it is estimated that considerably less than 1 MB will have to be transferred in the recognition process. For all types of cameraphones, the booking and reservation application requires additional data transfer in course of the conversation which results in a final confirmation from the event organizer to the cameraphone user, as described in the Use Case section above. Web pages for cameraphones would typically be very small, around 1-5 kB, and since there are four pages back and forth the ensuing traffic amounts to 4-20 kB. However, in case the image of the original advertisement is included in the final confirmation, which may be of interest to some users, 1 - 5 MB will be transferred either from the server back to the cameraphone, or as an e-mail to the end-user.
Digital camera requirements
Common ground for most of the conceivable applications involving a digital camera or cameraphone for capturing images containing symbolic information (such as alphanumeric text, bar code or spatial codes) is that the image information has to be interpreted in a sequence of operations according to Figure 4. This is true irrespective of whether the aim is translation of text on signs in a tourist application or ordering of goods from an ordering form in a mail-order advertisement.
The sequential steps in Figure 4 are as follows:
• End User - The end user captures an image to be used as input to the application.
• Prefiltering - Image operations to reduce noise and normalize contrast.
• Segmentation - Hypotheses are cut out from the image. A hypothesis can correspond to either whole lines of characters or other symbols or a single character or symbol.
• Recognition - For each hypothesis an estimate is made of the most probable character or symbol represented by it. Character or symbol hypotheses are then grouped into word hypotheses.
• Service - Finally, the interpreted information is used for the specific application requested.
As already explained with reference to Figure 3, depending on the performance of the mobile unit and the complexity of the application, these sequential steps can be performed either directly on the mobile unit or on a separate server accessed via the mobile network and Internet. Four typical system cases can be identified, representing different demands on i) the capacity of the mobile unit, ii) need for pre- installed software and databases and iii) the amount of data to be transferred when the application is used.
The four typical cases are marked in Figure 4 at the dashed lines. For each case, the part of the system to the left of the line is within the mobile camera and everything to the right is on external servers. Consequently, Type 1 corresponds to the most primitive model of a mobile unit, whereby all interpretation and processing is performed externally, while Type 4 corresponds to the most advanced high-end model, where all interpretation and processing is performed within the cameraphone. For instance, in a Type 3 cameraphone and system, the operations prefiltering, segmentation and recognition will be executed by the software within the mobile unit, while the service is performed on an external server.
The amount of data to be transferred between the sequential operations described above is very different, due to a consecutively higher degree of interpretation and therefore abstraction of the information. Consequently, the typical system cases described above will generate different amounts of network traffic. In the Type 1 system, the whole image is transferred (1-5 MB). In the Type 2 system, only hypotheses in the form of image data (0.2-1 MB) will be transferred to the recognition step on a server, but there will also be a feedback flow of information (10-100 KB) from the server to the cameraphone for improved recognition. The feedback might further be performed in multiple steps. In the Type 3 system, data is transferred as text (1-5 KB) to the server which performs the service, such as ordering goods via mail-order.
It is obviously quite feasible to implement the applications such, that the system will automatically identify to which category a given cameraphone belongs, so that users at any time can choose between the option to buy a cheaper cameraphone and pay higher traffic and service costs, or more advanced/more expensive mobile units, resulting in reduced traffic and service costs but a higher initial capital cost.
It should be observed, that the higher the degree of operations to be performed by the cameraphone is, the larger will be the requirement for initial download of software and databases either from the Internet after payment or from purchased memory cards. Thus, the more advanced mobile units will in many cases have a larger initial download via the network, while their continuous use of network capacity will be lower. On the other hand, the low-end cameraphones will require a much lower initial download of software, while the continuous need for bandwidth will be much higher.
Use case # 2
The Use case # 2 explains the application of cameraphones and digital cameras equipped with communication means to computers, for ordering of goods and services within the mail order and distance selling fields.
The distance selling field has long traditions. Worldwide, it accounts for about 10% of total sales volumes and its share is growing. In Europe, GB, Germany and the Nordic countries are leaders in the field of distance selling. In USA, the per capita volumes in this field are almost twice the European average.
When, in the following, reference is made to mail order and mail order companies, all distance selling activities which are not purely Internet based but involve catalogues, advertisements in magazines and newspapers, brochures and direct mailing operations aiming at attracting orders from customers via telephone, mail reply or Internet, are included.
Principal aspects of the mail order application for intelligent digital cameras
In the mail order application, the cameraphone or digital camera user switches the camera into the intelligent mode via a menu or by pressing a key and takes a snapshot of an offer from a book club, a mail order catalogue, an advertisement in a magazine or a news paper or some other printed medium. The offer is frequently, but not always, formalized as an order sheet or slip according to Figure 5, which shows an order sheet from a Swedish book club. As will be evident from the following, from the point of view of the mail order application as described herein, the format of the offer is not a determining factor and even offers on placards and bills in shop windows or on poster pillars can be captured by the cameraphone and transmitted, promoting impulse buying.
A precondition is, however, that the supplier has specified the offer in a suitable way, preferably by, e.g., an alphanumeric interpretable text or code or a bar code, such that the product or service offered can be identified by the camera, and that the buyer has filled in the order sheet or slip clearly enough to be interpretable by the OCR or HWR (Hand Writing Recognition) functions of the mail order application. This is not a critical constraint because in many cases, such as when ordering a book or a CD, the article number (which may be stated as a bar code) uniquely defines the product, relieving the buyer from any other effort than, e.g.., marking by a cross in a square what he requests, in case there are several options.
As in the booking & reservation application according to Use case # 1 , implementing the mail order application according to the invention will be very much facilitated, if the order slip or advertisement is equipped with a spatial digital code, which is easily detected by the camera and uniquely identifies the supplier and the specific application. Thereby, the cameraphone is immediately and automatically directed to the server executing the service. This aspect and the system architecture for realizing the application has already been explained in considerable detail in context with the booking & reservation application. Hence, for the Use case # 2, only the specifics of the mail order application need to be described.
Thus, thanks to the information contained in the unique i - and a - codes marked 1 and 2 in Figure 5, once the snapshot is taken, all subsequent functions will ensue fully automatically, i.e.,
the image is analyzed by the system with respect to all relevant information regarding the offer, be it products or services the user is identified either by the information stored in the cameraphone or by name and address printed by him on the order sheet/slip the information is transmitted to the supplier's data base and business system for automatic attendance, without manual handling upon processing in the supplier's system for checking of stock balance, prices, etc., a confirmation of the order is sent to the cameraphone, requesting acceptance in case of acceptance, a final confirmation is submitted to the buyer's cameraphone and, optionally and at the same time, to his e-mail address the legal aspect is satisfied by storage in the supplier's data base of the buyer's signature ("signature of file") or by suitable usage of electronic signatures and/or PIN codes
The parties involved in the mail order supply chain include the mail order company, the cameraphone user, the mobile network operator and the server operator. The introduction of the intelligent digital camera as a terminal for entering orders in the mail order field offers advantages to all parties involved, because: • the cameraphone user can instantly and with the minimum effort required to take a snapshot of an advertised offer and order goods or services wherever he is located, without going to his computer or a mailbox and without spending time on the telephone
• manual handling of order sheets and slips is reduced, as well as the even more expensive order handling by telephone; postage costs are eliminated; the cost of entering orders via the digital camera is much lower than the before mentioned costs, resulting in considerable total savings for suppliers
• the purchase behaviour of buyers can be influenced, such that impulse purchases are stimulated, increasing turn-over
• the legal aspects are similar to but in several ways easier to manage than those pertinent to Internet based sales
As in Use case # 1 , the server which is the link between the cameraphone, i.e., between the end user and the mail order company, may be operated by a network operator or by a third party.
As the server carries the service, the application can easily be tailored to the requirements of each particular mail order company. Consequently, the server operator can charge a fee for the service, commensurable to the advantages and cost savings it offers to the mail order company.
At a further development of the method according to the invention a three code arrangement, as specified below, further enhances the versatility of the targeting code method. The three- code- arrangement used in an Advertising and Booking Application (ABA) comprises in consecutive order and preferably arranged in one row with a quiet zone between and around the codes, the:
• i - code, which is also a logotype which instantly tells the camera phone user that the information he sees can be read and recognized by his unit, but also and according to the present invention is designed such that it is not only instrumental in helping the camera to localize the relevant (i.e., the code carrying) part of the image, but also carries important geometric information derived from its orientation and size, telling the ABA programme where to find the remaining codes and thus directing their segmentation
• z - code, which is specific for a given company or a specific branch or service of a company, such as, e.g., a given cinema in a multiple chain cinema company
• a - code, which is the offer specific code, e.g., a specific film or event advertised. From the above it follows that the i - and z - codes are invariant in the sense that they remain unchanged, when the message or offer advertised or otherwise communicated to the public varies. The i - code may be compared to the prefix www on the Internet, while the z - code may be regarded as equivalent to the name of a home page on the Internet.
Although the z - code may be chosen as a standard EAN barcode or as a standardized ECC 200 matrix code, it is much more important that the a - code, which will be frequently changed and must be in the hands of the suppliers/advertisers, is a standardized and publicly available code. Accordingly, it is a preferred embodiment of the code structure according to the present patent application, that the variable and offer specific code be an officially standardized and publicly or commercially available code.
It is furthermore an important characteristic of the invention that a code, preferably the z - code as defined above, is used - when printed on forms, such as, e.g., order sheets from mail order companies - to trigger the downloading into the MOA of the user's camera phone the correct template sheet against which the order sheet completed by the user is interpreted, before it is submitted to the server and data base of the mail order company. In this way, the MOA becomes entirely general, in spite of the fact that it requires the ability on the mobile phone's side to recognize a vast number of ever changing sheets.
The use of a targeting code to download templates into digital cameras and camera phones as a basis for the interpretation of filled in sheets has been illustrated with reference to a particular application, but it is not limited to said particular application and establishes a general method to facilitate the use of visual input via digital cameras to computers, particularly in context with camera phones.
According to the invention, it is an intentional property of the mail order application, as defined and understood here, that it not only relates to the traditional mail order field, but essentially comprises any business activities involving the offer of goods or services in a format which can be readily captured by a cameraphone. As an example, any subway passenger reading a newspaper like Metro or City in Sweden and attracted by an ad therein for a product or a service, can immediately and instantly under full mobility place his order by just taking a snapshot of the ad by means of his intelligent cameraphone, provided that the ad is suitably tagged as disclosed herein.
A further illustration of the versatility of the visual interface to computers, and an example of an important general class of camera phone applications according to my invention, is the case of paying train- or underground tickets, parking fees, etc. In this class of applications, a possibly illuminated entrance poster or other type of display with sufficiently large codes unique for the specific railway station, parking house, or similar public facility, is set up such that a train passenger, a car driver, etc., when passing into the station, the parking house, etc., can take a snapshot of the "entrance codes". When leaving the railway station at his destination or the parking house, the passenger and car driver will similarly take a snapshot of the "exit codes". Within the scope of the ABA, the entrance and exit cases will be registered by servers communicating with the camera phones of the passenger and car driver, whereby the type of service and the appropriate fee will be uniquely determined by the codes captured. For the precise identification of the type and location of the service utilized, the z - code as defined above is particularly well suited. The billing solution for the service in question is not part of this application, but several viable billing and payment solutions for mobile services are available and already practiced.
A further similar application, based on posters or displays carrying geographic information, is the ordering of, e.g., a taxi to a given location. By taking a snapshot of the poster which carries suitably designed codes, the camera phone will establish contact with the taxi company's server or switchboard, indicating that a cab is required to the location of the poster.
An important further extension of the ABA is the use of the camera phone as a ticket, implying that not only the booking of an event is made via the camera phone, as illustrated by the use case # 1 of my referenced Swedish patent application, but also that the confirmation from an event organizer, a cinema, an airline company, a supplier of goods, etc., is submitted to the user's camera phone in such a format, that it can serve and be read as a ticket, a dispatch note for collection of parcels, etc. This is achieved by sending from the organizer's or supplier's server to the user's camera phone not only a plain language confirmation, but also a code which can be read by suitable stationary or hand held scanners at the entrance to any event such as a cinema, a concert or a theatre, or any counter such as an airline check in counter or a postal office delivery counter. Tests have shown that a bitonal 2D-code on the display of a camera phone can be easily read and decoded by another mobile camera phone with the preinstalled ABA. This confirms that in spite of glare and reflections in the cover of the display, the fairly primitive optics of standard camera phones is sufficient to allow the implementation of the ticket function disclosed herein. With industrial type scanners, be it barcode scanners or matrix scanners, featuring good optics and optimal illumination techniques, quick and reliable reading of the ticket codes on the display of the camera phone is achieved.
It is further the subject matter of the present patent application, that the ticket codes be designed such that a given code displayed on the camera phone may represent several tickets or that booking can be made by one party but - at the request of the booking party - the tickets may be distributed to several camera phones, allowing uncoordinated admissions.
By the introduction of the ticket function as disclosed herein to the camera phone, the visual interface will encompass all aspects of the ABA, from responding to the advertisement, through booking, to ticketing and admission control, allowing considerable savings and efficiency gains on the supplier's side and freeing the camera phone users from queuing for tickets, etc.
Also, it is obviously allowed within the scope of the invention that the ticket function may be used with any type of mobile phone with a display and for any type of visually - be it by laser or other types of scanners , CCD or CMOS cameras or even the human eye of a third party - readable confirmation method, allowing transactions of a commercial type requiring that the mobile phone user is able to prove that he is entitled to some service or goods booked or prepaid by him in advance from the supplier of the service or goods, irrespective of whether said booking or prepayment has been made by the Internet, telephone or mobile phone, fax, mail or personally over the counter.
One of the most critical aspects in imaging by mobile digital cameras is blur, as blurred images make interpretation of codes difficult or even impossible. Blur may be caused by optical defects, such as bad focusing. However, motion blur is the most common cause for un-interpretable codes. It is therefore important to have diagnostic means in the mobile camera, which will give he user feedback on the quality of pictures taken, making it possible for the user to improve his imaging technique. An approach for detecting blur in images is based on the fact that blurred images frequently display various degrees of sharpness in perpendicular directions. In a blurred direction gradients are small, whereas in a non-blurred direction gradients are large. Thus, if a given picture exhibits marked differences of gradients in two perpendicular directions, it is very likely that the image is affected by motion blur. Detection of gradients is therefore a method which allows prediction of motion blur, The motion blur diagnostics introduced by the method according to the present invention is based on this gradient comparison technique.
According to yet another aspect of the invention the images captured by the mobile camera can be used either for connecting instantly to the web site of interest or can be stored as a bookmark for later action. This can be important if, e.g., a decision to order tickets or similar is subject to approval by a third party, as a family member. Practically, the bookmark function is achieved by either storing the captured image as a special file or by just postponing the submission of the image temporarily stored in the application.
Although it is beyond the subject matter of the present patent application, it can be shown that the economic implications of making it possible to use, in a realistic and robust way as disclosed herein, the "eye" of digital cameras and cameraphones, are very large, because effortless and truly mobile access to the Internet is made possible, thereby even further enhancing the continuously increasing importance of Internet based services.
The invention has been described with reference to two practical cases of great significance and in its presently foreseen best modes. However, it is clear that the invention is susceptible to changes of mode of operation and embodiment, all within the ability of persons skilled in the art and the frame of normal technical development, and within the scope of the invention as set forth in the appended claims.