US6793128B2 - Face photo storage system - Google Patents

Abstract

A face photo storage system is disclosed. A digital camera takes one or more photos of a person in response to the detection of an event, such as a cash register having rung up a sale to the person, the person entering the premises, and so on. The best photo of the person that includes the best picture of the person's face is determined. A database stores this photo, along with at least a current date in which the photo was taken. The database may also store the current time at which the photo was taken, a transaction record of the person's purchase, and so on. The invention may be used in conjunction with or in lieu of a video surveillance system.

Description

FIELD OF THE INVENTION

The invention relates generally to recording persons, such as accomplished by video surveillance systems, and more particularly to storing a photo of a face of a person in response to detecting the person or in response to another event.

BACKGROUND OF THE INVENTION

Video surveillance systems, such as those used in small business settings like convenience and other stores, usually employ one or more video cameras. The video cameras are typically located in strategic positions within the given environment. For example, in a store, there may be one video camera aimed at the entrance to the store, and another video camera aimed at the counter of the store at which sales are rung up, such as where a cash register is located.

These surveillance systems may have the capability of recording on videotapes all people entering, making a purchase, and/or leaving the store. If a theft, robbery, or break-in occurs, the videotapes can be reviewed to determine whether a good image of the perpetrator has been recorded. This image can then be compared to mug shot databases maintained by law enforcement, circulated to news media to display during newscasts and in newspapers, and so on.

However, such surveillance systems have some disadvantages. For example, if a theft has occurred, but the exact time of which is not known, several days worth of videotape may have to be reviewed to learn who has perpetrated the theft. This can be a very time-consuming and inconvenient process. Furthermore, the image of the perpetrator recorded on the videotape may be of poor quality, making a positive identification from the image difficult if not impossible.

For these and other reasons, there is a need for the present invention.

SUMMARY OF THE INVENTION

The invention relates to a face photo storage system. A system of one embodiment includes a digital camera, a detection mechanism, a face detection and selection mechanism, and a database. The digital camera takes one or more photos of a person in response to the detection mechanism detecting an event, such as a cash register having rung up a sale to the person, the person entering the premises, and so on. The face detection and selection mechanism determines the best photo of the person that includes the best picture of the person's face. The database stores this photo, along with at least a current date in which the photo was taken. The database may also store the current time at which the photo was taken, a transaction record of the person's purchase, and so on.

A method of one embodiment of the invention takes one or more photos of the person, and determines the best photo that includes the best picture of the person's face. The method stores the best photo with at least the current date in the database. A computer-readable medium of one embodiment has instructions stored thereon to perform a method. This method detects an event, and in response thereto causes a digital camera to take one or more photos of the person. The method determines the best photo of the person that includes the best picture of the face of the person, and causes this best photo to be stored with at least the current date in the database.

At least some embodiments of the invention provide for advantages not found within the prior art. Rather than having to review several days worth of videotape from a video surveillance system, a user only has to scroll through the various faces stored in the database during a period of time, which is likely to be less time-consuming. Because the system determines and stores the best photo of each person's face, the images are likely to be of better quality than those that may be stored on videotape. Where the transaction record of each person's purchase is also recorded, the user may be able to search for the photo of the face of a desired person based on what has been purchased by him or her. Finally, the faces of people stored in the database may be able to be matched against databases of mug shots maintained by law enforcement.

At least some embodiments of the invention may be employed in conjunction with or in lieu of a video surveillance system. Still other advantages, embodiments, and aspects of the invention will become apparent by reading the detailed description that follows, and by referring to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an example environment in conjunction with which embodiments of the invention may be used.

FIG. 2 is a diagram of a system according to an embodiment of the invention.

FIG. 3 is a diagram of another system according to an embodiment of the invention.

FIG. 4 is a diagram of a method according to an embodiment of the invention.

FIG. 5 is a diagram of an example application in conjunction with which embodiments of the invention may be used.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of exemplary embodiments of the invention, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific exemplary embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Other embodiments may be utilized, and logical, mechanical, and other changes may be made without departing from the spirit or scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims.

Example Environment and Overview

FIG. 1 shows an example environment in conjunction with which embodiments of the invention may be implemented. More specifically, FIG. 1 shows the premises of astore100. Thestore100 includes amain shopping area102, anentry area104, and acheckout area106. Theentry area104 has adoor108 to allow people to enter thestore100. There is avideo camera110 that is part of a video surveillance system, and adigital camera112 that is part of an embodiment of the invention. Thecheckout area106 includes acash register counter114. There is avideo camera116, also part of the video surveillance system, and adigital camera118, which is also part of the embodiment of the invention.

When a person walks through thedoor108 into thestore100, he or she is under video surveillance by thevideo camera110. A detection mechanism causes thedigital camera112 to take one or more photos of the person. Similarly, when the person makes a purchase at thecash register counter114, he or she is under video surveillance by thevideo camera116, and a detection mechanism causes thedigital camera118 to take one or more photos of the person. The embodiment of the invention can be used in conjunction with the video surveillance system of which thevideo cameras110 and116 are a part, or in lieu of them.

The set of pictures taken by thedigital camera112 and the set taken by thedigital camera118 are processed to determine a best photo of each set, including a best picture of the face of the person. Preferably along with the current date and time, as well as the transaction record of the purchase made by the person at thecounter114 in the case of thedigital camera118. The best photos from eachdigital camera112 and118 are stored in a database. These photos can then be used for varied purposes and applications, as described later in the detailed description of the invention.

Systems

FIG. 2 shows asystem200 according to an embodiment of the invention. Thesystem200 may be used at theentry area104 of FIG. 1, for instance. Amotion detector202 detects the presence of a person. Themotion detector202 may be thevideo camera110 of FIG. 1, where a change in the field of view of thevideo camera110 from an earlier frame to a later frame indicates that a person has entered thestore100 of FIG.1. Themotion detector202 may also be thedigital camera112 of FIG.1. Themotion detector202 is generally a type of detection mechanism. The presence of the person is referred to generally as an event, and is one type of such an event.

In response to detection of the presence of a person, themotion detector202 causes thedigital camera204 to take a number ofphotos206a,206b, . . . ,206nof the person. Thedigital camera204 may be thedigital camera112 of FIG.1. Thephotos206a,206b, . . . ,206nare input into face detection andselection software208. The face detection andselection software208 typically runs on a computer, and is generally a type of face detection and selection mechanism. The face detection andselection software208 selects thebest photo210 of thephotos206a,206b, . . . ,206nof the person that includes the best picture of the person's face. The “best” picture in this case is generally defined as the most representative picture of the person's face, or the picture of the person that would be most easily recognized.

It is noted that, in at least some embodiments of the invention, the face detection andselection software208 does not identify the person in thephoto210, but only selects thebest photo210 of the person in thephotos206a,206b, . . . ,206n. Thebest photo210 is non-restrictively defined as thephoto210 that shows the best picture of the person's face. Thebest photo210 may be compressed so that it does not occupy a large amount of storage space. For example, thebest photo210 may be compressed according to a lossy compression scheme, such as the Joint Photographic Experts Group (JPEG) standard. Furthermore, thebest photo210 may also be encrypted, according to any of a number of different encryption schemes, to allay potential privacy concerns.

Themotion detector202 also causes the clock/calendar software212 to stamp the current time anddate214 on thebest photo210. This is accomplished so that the time anddate214 at which thebest photo210 was taken by thedigital camera204 can later be referenced. Thebest photo210, with the current time anddate214, is sent over theInternet216 for storage at a service provider'sdatabase218. It is thus said that the face detection andselection software208 and/or the clock/calendar software212 are communicatively coupled to thedatabase218 over theInternet216. TheInternet216 is one type of network. The invention may be implemented in conjunction with other or additional networks as well, such as intranets, extranets, local-area networks (LAN's), wide-area networks (WAN's), and so on.

Thebest photo210 and the current time anddate214 may be uploaded to thedatabase218 as soon as thephoto210 is determined. Alternatively, they may be sent on a batch basis, either every so often, or once a certain number of best photos have been taken. Thedatabase218 as shown in FIG. 2 is remote, in that it is connected to thesystem200 over theInternet216. However, thedatabase218 may be a local database as well, residing in the same premises as thedigital camera204, themotion detector202, and so on. Thedatabase218 also stores other best photos and the dates at times at which they were taken, besides thephoto210 and the time anddate214.

Thesystem200 can be provided as a turnkey operation, in which the owner of the store or other premises in which it is installed does not have to have computer experience. Once thedigital camera204 and themotion detector202 are situated, and the face detection andselection software208 and the clock/calendar software212 are installed, the owner does not have to otherwise maintain thesystem200. Rather, thesystem200 is maintained by the service provider that also maintains thedatabase218. Once thebest photo210 and the current date andtime214 are stored in thedatabase218, they can be used for a variety of different purposes and applications, as is described later in the detailed description.

FIG. 3 shows asystem300 according to another embodiment of the invention. Thesystem300 may be used at thecheckout area106 of FIG. 1, for instance. When a sale is rung up to a person at the cash register302, this causes thedigital camera204 to take the number ofphotos206a,206b, . . . ,206nof the person. The cash register302 may be located at thecash register counter114 of FIG. 1, whereas thedigital camera204 may be thedigital camera118 of FIG.1. Alternatively, amotion detector202, such as thevideo camera116 of FIG. 1, can be used to detect the presence of the person. The cash register302 is also considered as a type of detection mechanism. The ringing up of a sale to the person is also referred to generally as an event, and is one type of such an event.

Thephotos206a,206b, . . . ,206nare again input into the face detection andselection software208, which selects thebest photo210 therefrom that includes the best picture of the person's face. When the sale is rung up at the cash register302, this also causes the clock/calendar software212 to stamp the current time anddate214 on thebest photo210. Thebest photo210, along with the current time anddate214, and a transaction record of thesale304, is sent over theInternet216, or a different or additional network(s), for storage at the service provider'sdatabase218. Thetransaction record304 may include the credit card number used by the person to make the purchase, what was purchased by the person, and so on.

Like thesystem200 of FIG. 2, thesystem300 of FIG. 3 can be such that thebest photo210, the current time anddate214, and thetransaction record304 may be uploaded to thedatabase218 individually or on a batch basis. Thedatabase218 may110 be a local database instead of a remote database. Thesystem300 of FIG. 3 may also be provided as a turnkey operation to the owner of the store or other premises by the service provider. Thesystem300 may be used in conjunction with thesystem200 of FIG. 2, or as a separate system. Once thebest photo210, the current time anddate214, and thetransaction record304 are stored in thedatabase218, they can be used for a variety of different purposes and applications, as is described later in the detailed description.

Method and Example Applications and Uses

FIG. 4 shows amethod400 according to an embodiment of the invention. Themethod400 may be performed in conjunction with either thesystem200 of FIG. 2, thesystem300 of FIG. 3, both thesystems200 and300, or neither system. At least some parts of themethod400 may be performed by the execution of instructions by a processor that are stored on a computer-readable medium, such as a memory, a floppy disk, a CD-ROM, or another computer-readable medium.

First, a trigger event occurs (402). The trigger event may be the detection of the presence of a person, the ringing up of a sale to the person on a cash register, and so on. A number of photos are then taken of the person (404). From these photos, a best photo is selected, or determined, that includes the best picture of the person's face (406). Other data may optionally be attached to the best photo, such as the current time and date, a transaction record, and so on (408). The best photo and the other data are stored in a database, such as the service provider's database (410). This may be accomplished by sending the best photo and other data over a network, such as the Internet, for storage in a remote database. The photos stored in the database may then be used as desired, for a variety of different applications and purposes (412).

FIG. 5 shows asystem500 of one example application and use for the photos stored in the database. Thephoto502 includes the face of a person. Thesystem500 determines whether this person has visited the store or other premises for which the database ofphotos218 has been maintained as has been described. For example, thephoto502 may include the face of a person that is wanted by law enforcement, as a possible suspect in a string of robberies. Face-matchingsoftware504, which usually runs on a computer, determines whether the face in thephoto502 matches any of the faces of the photos stored in thedatabase218. Thesoftware504 then provides anoutput506 indicating its conclusion. If a match was found, thesoftware504 can provide the date and time at which the person visited the store or other premises, and possibly the transaction record of a purchase made by this person.

At least some embodiments of the invention are amenable to other types of applications and uses. For example, a photo of a face of a person that is stored in thedatabase218 maybe matched by the face-matchingsoftware504 against a database of mug shot photos maintained by law enforcement. The photo may be of a person that has stolen something from the store or other premises. Therefore, the face in the photo is desirably matched against the photo database maintained by law enforcement, to determine whether the person has committed other crimes in the past.

Another example is to match a credit card submitted by a customer for the purchase of an item with the photos of the person(s) who previously used the same credit card. A customer, for instance, may give the clerk behind thecounter114 of FIG. 1 a credit card. When the clerk swipes this card for validation purposes, the photos of the person(s) who previously used this credit card at the store are displayed to the clerk. The clerk can then make a quick decision as to whether the current customer is the same person shown in the photos, or if, for example, the credit card was stolen by the customer.

Applications and uses that are not related to law enforcement and security are also contemplated. For example, when thebest photo204 is selected, different face-matching software may immediately compare thebest photo204 with the photos stored in thedatabase218. For any matches that are found, the software may display the associated dates, times, and transaction records to the storeowner, either in real-time, or at a later time. The owner of the store or other premises can, for instance, determine whether a repeat customer has entered his or her store, and view that customer's previous purchases. The owner can use this information while the customer is in the store to show the customer additional attention, in an attempt to increase sales.

It is noted that, although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that any arrangement is calculated to achieve the same purpose may be substituted for the specific embodiments shown. For example, other applications and uses of embodiments of the invention, besides those described herein, are amenable to at least some embodiments. This application is intended to cover any adaptations or variations of the present invention. Therefore, it is manifestly intended that this invention be limited only by the claims and equivalents thereof.

Claims (20)

I claim:

1. A surveillance system comprising:

a digital camera;

a detection mechanism to cause the digital camera to take one or more photos of a person, in response to detection of an event;

a face detection and selection mechanism to determine a best photo of the one or more photos of the person, the best photo including a best picture of the face of the person from which the person is most easily recognized; and,

a database to store the best photo of the face of the person with at least a current date in which the best photo was taken, the database also storing a plurality of best photos of faces of people.

2. The system ofclaim 1, wherein the detection mechanism comprises a motion detector to detect presence of the person, such that detection of the presence of the person causes the digital camera to take the one or more photos.

3. The system ofclaim 1, wherein the detection mechanism comprises a video camera, such that a change in a field of view of the video camera from an earlier frame to a later frame of the video camera causes the digital camera to take the one or more photos.

4. The system ofclaim 1, wherein the detection mechanism comprises a cash register, such that ringing up a sale to the person on the cash register causes the digital camera to take the one or more photos.

5. The system ofclaim 4, wherein the database further stores a transaction record of the sale.

6. The system ofclaim 1, wherein the detection mechanism is the digital camera itself.

7. The system ofclaim 1, wherein the face detection and selection mechanism comprises face detection and selection software running on a computer.

8. The system ofclaim 1, wherein the face detection and selection mechanism at least one compresses and encrypts the best photo of the face before the database stores the best photo of the face.

9. The system ofclaim 1, wherein the face detection and selection mechanism is communicatively coupled to the database over a network.

10. The system ofclaim 1, wherein the database further stores a current time at which the best photo was taken.

11. A surveillance method comprising:

taking one or more photos of a person;

determining a best photo of the one or more photos of the person, the best photo including a best picture of a face of the person from which the person is most easily recognized; and,

storing the best photo of the face of the person with at least a current date in which the best photo was taken in a database also storing a plurality of best photos of faces of people.

12. The method ofclaim 11, initially comprising detecting an event, in response to which the one or more photos of the person are taken.

13. The method ofclaim 12, wherein detecting the event comprises detecting a presence of the person.

14. The method ofclaim 12, wherein detecting the event comprises detecting a sale to the person having rung up on a cash register.

15. The method ofclaim 14, wherein storing the best photo of the face of the person with at least the current date comprises storing the best photo of the face of the person with at least the current date and a transaction record of the sale.

16. The method ofclaim 11, wherein storing the best photo of the face of the person with at least the current date comprises storing the best photo of the face of the person with at least the current date and a current time at which the best photo was taken.

17. The method ofclaim 11, further comprising, alter determining the best photo, sending the best photo of the face of the person with at least the current date to the database over a network.

18. A computer-readable medium having instructions stored thereon to perform a surveillance method comprising:

detecting a event;

in response to detecting the event, causing a digital camera to take one or more photos of a person;

determining a best photo of the one or more photos of the person, the best photo including a best picture of a face of the person from which the person is most easily recognized; and,

causing the best photo of the face of the person to be stored with at least a current date in which the best photo was taken in a database also storing a plurality of best photos of faces of people.

19. The medium ofclaim 18, wherein detecting the event comprises one of: detecting a change in a field of view of a video camera from an earlier frame to a later frame, and detecting a sale to the person having rung up on a cash register.

20. The medium ofclaim 18, wherein causing the best photo of the person to be stored with at least the current date comprises sending the best photo of the face of the person and at least the current date to the database over a network.

Images