SYSTEM, METHOD AND APPARATUS FOR BETTER MANAGEMENT OF A PREMISE USING SENSOR AND CCTV ANALYTICS
FIELD OF THE INVENTION
The present invention relates to the field of business automation and better management of a retail and commercial premise. More particularly, the present invention relates to an effective and efficient system, method and apparatus for better management of a premise using video and sensor analytics.
BACKGROUND OF THE INVENTION
Technology has improved a variety of processes in the retail and commercial sector. However, many tasks are still performed manually that today's technological advances are able to automate. For example, a typical retail and commercial premise (e.g., shopping mall, grocery store, retail outlets, quick service restaurants, department store, banking, manufacturing units, hotels, market research government and other brick-and-mortar setups etc.), is still dependent on conventional techniques which involve analysis of data collected by human observers who, typically, move around and manually record information relevant to monitoring retail business processes for example an employee manually keeps count of the footfalls in a retail premise.
In other implementations many retail businesses conventionally employ closed- circuit television (CCTV) cameras on the premises. The video from the CCTV cameras are observed by one or more humans and/or recorded for later observation by one or more human observers. However, to monitor or review the video provided by all available CCTV cameras might require a substantial number of humans and a substantial expense. As such, placing CCTV cameras in all desirable locations of a retail business and contemporaneously or non- contemporaneously monitoring all the video from the CCTV cameras is not practical. As such, conventional video analysis systems still require employees for control and decision-making tasks to improve the efficiency of the retail business model. However, the need for human decision-making in these systems leads to inconsistent decisions and are dependent on that person's expertise.
Considering the dynamic nature of retail business, it is imperative for a business to have a pulse of the customer traffic patterns across a premise to optimize their operations and drive business outcomes, such as to have information on:
• how many customers/shoppers are coming to the premise
· what are the footfall trends at various times in the premise
• which are the most visited areas of the premise
• which areas are dead areas are dead areas within the premise
• how to improve customer flow within the store
• why some sections are not selling etc.
Further, the retail stores or brands like to encourage customers to repeatedly visit their businesses so that retail stores or brands can have more opportunities to increase business during these repeat visits. A variety of customer loyalty programs are used for this purpose. Retail stores or brands, which have already automated their business operations with a computer system, can ask a customer to fill out an application form and issue a membership card (or an equivalent instrument) to the customer. The membership card bears identification information (e.g., an account number) to identify the customer. Based on the member identification information of the customer, the computer system tracks the activities of the customer and provides future discounts, rebates or other types of rewards to the customer. If the customer forgets to bring his/her membership card during a transaction, his/her home phone number can be used to identify the member account number so that purchase activity is recorded and rewards may be issued accordingly.
A problem with this approach is that many consumers do not like to release personal information to merchants for privacy concerns. Furthermore, some consumers who are moving at a fast pace do not want to spend time filling out application forms. Moreover, because many merchants are each offering their respective membership reward programs, consumers are tired of having to fill out applications over and over again. In addition, it is not convenient for consumers to carry many membership cards or instruments. Some of the contemporary systems have another problem in that they collect consumers details or personally identifiable information with or without consumer's consent which can be misused or mismanaged thereby leading frauds and other privacy violations, as the details or information is analyzed through human interface making it very difficult to control their further use.
Taking into account the foregoing, the brick-and-mortar retailers today only get to track the buying patterns of customers who have bought before. While the Customer relationship management (CRMs) track the post-billing details, they fail to offer visibility into their overall traffic patterns, the true behaviour of their loyal customers - their recency and frequency; when did the customer pass the store and not enter? When did the customer enter the store and not purchase?; and what was the reason for no purchase? Added to this, the retailers have no channel or means of knowing when their repeat customers are in the vicinity? How to get them back in the store? Which could be the best time to lure them into the store?
Video and sensor analytics technology, the related apparatus, method and system according to the present invention is more than capable of addressing and resolving the issues discussed above.
OBJECTS OF THE INVENTION
The present invention relates to an apparatus and the related method and system for capturing customer behaviour data and providing a "last-mile connectivity" platform to retailers to connect with their past and probable customers. When a customer/shopper is at or in vicinity of the store, his/her shopping journey and behaviour can provide important indicators to retailers to connect with them and drive them into making a purchase. The customers/shoppers get personalized digital marketing content in real-time on their mobile devices when they are within the store or in vicinity, and interact in real-time with the retail store or brand.
Thus, a primary object and advantage of the present invention is to address and overcome the problems cited in the prior art.
Another object and advantage of the present invention is to provide an improved system, method and apparatus for better management of a premise using sensor and CCTV feeds and analytics.
Another object and advantage of the present invention is to use video feeds from a CCTV camera for analysing customer traffic patterns across a premise to optimize their operations and drive business outcomes. Another object and advantage of the present invention is to take a copy of video feeds stream from the store's existing CCTV infrastructure without any human intervention and convert them into meaningful shopper data for the retail stores or brands, without identifying any shopper. This ensures to maintain the privacy of the customer whilst providing all relevant data.
Another object and advantage of the present invention is to deal with the problem of privacy violations through data collection and analysis as discussed above.
Another object and advantage of the present invention is to provide a "last-mile connectivity" platform to the retailers to connect and engage with their probable customers and send personalized digital marketing content in real-time to the mobile devices of these shoppers/customers when they are within the store or in vicinity. The device, upon receiving the signals, makes real-time correlations with the existing CRM, online demographics database, and/or existing CCTV video feeds, and sends out triggers to the retailer to push any amount or type of targeted marketing content to the customers. Another object and advantage of the present invention is to provide a method and system which enables shoppers/customers to interact in real-time with the retail store or brand. The shoppers can share their experiences, demands and issues so that the retailers know just in time the reason behind the "purchase" or "no purchase", or the customer's attitude towards the retail store or brand.
Another object and advantage of the present invention is to provide an improved system, method and apparatus to demonstrate the customer traffic pattern throughout the day, week, month and year for better management of a premise.
Another object and advantage of the present invention is to provide an improved system, method and apparatus for better management of a premise which can distinguish the customer traffic pattern by way of gender or age demographics, individual or groups such as families, or other characteristics/demography.
Another object and advantage of the present invention is to provide an improved system, method and apparatus for better management of a premise wherein actual shoppers/customers can be segregated from staff, security staff, housekeeping staff to get a more accurate footfall count.
Another object and advantage of the present invention is to provide an improved system, method and apparatus for better management of a premise wherein customer traffic pattern in a selected zone can be correlated with the overall footfall/dwell times of the premise to find if this is a high-impact location for the premise.
Another object and advantage of the present invention is to provide a sensor which detects a mobile phone of a shopper in the vicinity in a premise. Once detected the sensor transmits a unique number to a web server via a network which is stored therein with the date and time stamp. The same unique number when comes from multiple locations with different time stamps helps in identifying the path/journey taken by a shopper. Yet another object and advantage of the present invention is to generate metrics to measure key-performance index for a premise which can be used to measure, not being limited to, staff efficiency, gross-margin-return-on-dwell, category wise performance and trial percentage, queue disbursements etc.
SUMMARY OF THE INVENTION
One aspect of the invention relates to a system for better management of a premise using video and sensor analytics comprising: a CCTV camera which is used to grab a video feed, a digital video recorder (DVR) which records the feeds of the CCTV, one or more sensors to process video feed obtained by the CCTV and to convert the video feed into metadata; a network connecting the sensor with the store DVR to fetch and process the stored video feeds in real-time, a webserver deployed with a web service connected to the network, a database server connected to the network which in turn connected to a web dashboard configured to process the metadata for generating analytics reports. The sensor is configured to detect any Wi-Fi or Bluetooth enabled handheld device within the range of the sensor. The sensor is configured to track the unique media access control (MAC) ID of the handheld device. The sensor is configured to process the fetched video feed and MAC IDs and to convert the video feed and MAC IDs into metadata.
The sensor removes all personal idenifiable information (PI I) and only uploads the metadata on a web server or cloud server thus maintaining complete privacy of the data and ensuring that no Pll from the video feed goes out of the retail store.
Another aspect of the invention relates to an apparatus for better management of a premise using video and sensor analytics comprising: one or more sensors comprising: a processing unit used to convert a video feed from a CCTV infrastructure of the premise and the Media Access Control (MAC)-IDs of the Bluetooth and Wi-Fi enabled devices into metadata within the range of the sensor; a Wi-Fi unit connected to the processing unit being configured to detect the MAC- IDs of the Wi-Fi enabled devices within the range of the sensor; a Bluetooth unit connected to the processing unit being configured to detect the MAC-IDs of the Bluetooth enabled handheld devices within the range of the sensor, a network connecting the processing unit of the sensor to a computing resources and cloud storage units for further processing of the metadata to analyse and generate the analytics report for the client or retail store.
The CCTV infrastructure includes cameras and digital video recorder (DVR) connected to the network which includes: an internet; an intranet; a local area network (LAN); a wide area network (WAN); and a combination of networks, such as an internet and an intranet. The processing unit is configured to submit the metadata to web server or cloud server using the network. The processing unit is configured to save the collected and processed metadata in the in-built internal storage database or other local storage. Yet another aspect of the invention relates to a method for better management of a premise using video and sensor analytics, the method comprising the steps of: switching on the sensor, the CCTV camera, DVR and the network; prompting the sensor to access the video feed from the DVR and converting the video feed into metadata; collecting the MAC-IDs of the handheld devices/smartphones of the customers detected within the range of the sensor; forwarding the metadata and the MAC-IDs to a web server or cloud server using the network; performing the proper verification that there is no data loss during the submission to web server or cloud server; forwarding the data including the metadata and the MAC-IDs received by the web server to the app server; employing the app server to combine the received data including the metadata and the MAC-IDs with the store configuration data; processing and analysing the combined data with the help of a business intelligence software in order to create the relevant analytics reports for the clients or store as per the need of clients or store; providing a prepared report to the clients or store which can be accessed on an interactive web-dashboard.
The method also includes steps of determining and preparing the personalized content for the specific consumers on the basis of selected MAC-ID and providing the personalized content to the web server over an interactive web-dashboard which can be accessed on handheld devices/smartphones of the customers while he /she is still within the range of the sensor.
In an another embodiment, a method for better management of a premise using video and sensor analytics is provided wherein the method comprising the steps of: switching on the sensor, the CCTV camera, DVR and the network of the premise/store; prompting the sensor to access the video feed from the DVR and converting the video feed into metadata; collecting the MAC-IDs of the handheld devices/smartphones of the customers detected within the range of the sensor; forwarding the metadata and the MAC-IDs to a web server or cloud server using the network; forwarding the metadata and the MAC-IDs received by the webserver to the database server; applying the logic of personalized content specific to the identified customer based on their MAC-ID to generate a personalized message for the targeted/identified customer; sending the generated personalized data or message back to the web-server; rendering the personalized data or message onto the web browser which can be accessed by the customer on their handheld device.
The reports of personalized data or messages sent to the identified customers are made available to the client or stakeholders on the web dashboard which can be monitored and changed by the client or stakeholders or premise ownwer as per their need.
In another embodiment, a method for better management of a premise using video and sensor analytics is provided wherein the method comprising the steps of: switching on the sensor, the CCTV camera, DVR and the network of the premise/store; prompting the sensor to access the video feed from the DVR and converting the video feed into metadata; collecting the MAC-IDs of the handheld devices/smartphones of the customers detected within the range of the sensor and accordingly calculates the footfall to the store/premise; qualifying the footfall count into time spent by the customer inside the store/premise; determining the percentage of customers or shoppers and the time spent by them inside the store end up buying a product; benchmarking the conversion potential that the retail store must have and prepare a report thereon and making the benchmark reports made available to the client or retail store/premise owner.
In yet another embodiment, a method for better management of a premise using video and sensor analytics is provided wherein the method comprising the steps of: switching on the sensor, the CCTV camera, DVR and the network of the premise/store; prompting the sensor to access the video feed from the DVR and converting the video feed into metadata; collecting the MAC-IDs of the handheld devices/smartphones of the customers detected within the range of the sensor; extending the store/premise network with a capability for customers/shoppers to latch on to it; making the premise/store's Wi-F network available on the customer's handheld device; prompting the customer to latch onto the store network by agreeing to the terms and conditions; connecting the customer to the retail store or brand once the connection is established; forwarding coupons, vouchers, personalized content, shopping related information, information relating to discounts and offers to the customer.
In one embodiment, the sensor detects a mobile device of a shopper in the vicinity, not being limited to, 12 to 15 meters. Once detected, the sensor transmits a unique number to a server via a network, not being limited to, e.g. Wi-Fi or GSM or CDMA, and the unique number is stored in the server along with the date and time stamp. The same unique number when comes from multiple locations with different time stamps helps in identifying the path/journey taken by a shopper. Thus, apparatus and the related method and system of the present invention help the retailers to extend their CRM, analytics and marketing solutions to capture pre-billing trends, connect their online and offline retail channels, and engage shoppers in the right manner, just in time, thus increasing the probability of the customers reaching the billing counter.
The apparatus of the present invention enables customer who are in-store to connect with the various brands via store internet and also tracks and analyses the shopper/customer behaviour via the store's existing CCTV infrastructure. It connects the shoppers/customers with the retail stores or brands in real-time while shoppers are at the store. This enables the retails stores or brands to do any of the following : a. Personalized curated content for the shopper/customer b. e-Vouchers
c. Campaigns
d. Information
e. Feedback
f. Digital content
g. 3rd party content
BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS The above, as well as additional objects, features and advantages of the present invention, will be better understood through the following illustrative and non- limiting description of preferred embodiments of the present invention, with reference to the appended drawings, wherein: FIG 1 illustrates an exemplary embodiment of the sensor used in the invention for better management of a premise using video and sensor analytics.
FIG. 2 illustrates an exemplary flow diagram showing an example process explaining how the hardware components within the sensor are interconnected and work together.
FIG 3 illustrates an exemplary embodiment of the system for better management of a premise using video and sensor analytics wherein the sensor works to deliver "video analytics" module.
FIG 3A illustrates another exemplary embodiment of the system for better management of a premise using video and sensor analytics wherein the sensor works to deliver "engage" module. FIG 4 illustrates a flow diagram an example process of how the sensor interacts with other components of the system depicted in FIGS. 3 & 3A. FIG 5 illustrates a flow diagram of an example process explaining the customer engagement opportunities in terms of receiving personalized messages and their feedback to the retail store or brands, shopping experience or on other factors. FIG 6 illustrates a flow diagram of an example process explaining - how retailer stores can establish the benchmark conversion for their store(s).
FIG 7 illustrates a flow diagram of an example process explaining - how retailer stores can build a two-way customer engagement platform using proximity marketing and customer feedback.
FIG 8 illustrates a flow diagram of an example process explaining - how Quick Serve Restaurants can reconcile transactions to minimize billing errors or frauds.
DETAILED DESCRIPTION OF THE INVENTION
Exemplary embodiments of the invention are discussed in detail below. While specific exemplary embodiments are discussed, it should be understood that this is done for illustration purposes only. In describing and illustrating the exemplary embodiments, specific terminology is employed for the sake of clarity. However, the invention is not intended to be limited to the specific terminology so selected. A person skilled in the relevant art will recognize that other components and configurations may be used without parting from the spirit and scope of the invention. It is to be understood that each specific element includes all technical equivalents that operate in a similar manner to accomplish a similar purpose. The examples and embodiments described herein are non-limiting examples. The following definitions are applicable throughout (including above).
The use of terms "sensor" or "ShopR360" in the description or drawings refer to the sensor only. "ShopR360" is the proprietary trademark of the applicant Apertura Data Tech Pvt Ltd.
As used throughout this description, the word "may" is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words "include," "including," and "includes" mean including, but not limited to.
The use of term "premise" may refer to retail store, commercial store, store, brick- and-mortar retailers, client, clients shopping mall, grocery store, retail outlets, quick service restaurants, department store, banking, manufacturing units, hotels, market research government and other brick-and-mortar setups etc. or the like.
"Video" may refer to motion pictures represented in analog and/or digital form. Examples of video may include: television; a movie; an image sequence from a video camera or other observer; an image sequence from a live feed; a computer- generated image sequence; an image sequence from a computer graphics engine; an image sequence from a storage device, such as a computer-readable medium, a digital video disk (DVD), or a high-definition disk (HDD); an image sequence from an IEEE 1394-based interface; an image sequence from a video digitizer; or an image sequence from a network.
A "video feed" may refer to some or all of a video.
A "video camera" or "CCTV" may refer to an apparatus for visual recording. Examples of a video camera may include one or more of the following: a video imager and lens apparatus; a video camera; a digital video camera; a color camera; a monochrome camera; a camera; a camcorder; a PC camera; a webcam; an IP Camera (wired as well as wireless); an infrared (IR) video camera; a low-light video camera; a thermal video camera; a closed-circuit television (CCTV) camera; a pan, tilt, zoom (PTZ) camera; and a video sensing device. A video camera may be positioned to perform surveillance of an area of interest.
"Video processing" may refer to any manipulation and/or analysis of video, including, for example, compression, editing, surveillance, and/or verification.
A "frame" may refer to a particular image or other discrete unit within a video.
An "apparatus" may refer to one or more sensors, one or more computers and/or one or more systems that are capable of accepting a structured input, processing the structured input according to prescribed rules, and producing results of the processing as output. Examples of a computer may include: a distributed computer system for processing information via computer systems linked by a network; an optical computer; a quantum computer; a biological computer; two or more computer systems connected together via a network for transmitting or receiving information between the computer systems; and one or more apparatus and/or one or more systems that may accept data, may process data in accordance with one or more stored software programs, may generate results, and typically may include input, output, storage, arithmetic, logic, and control units.
"Software" may refer to prescribed rules to operate a computer. Examples of software may include: software; code segments; instructions; applets; precompiled code; compiled code; interpreted code; computer programs; and programmed logic.
A "computer readable medium" may refer to any storage device used for storing data accessible by a computer. Examples of a computer-readable medium include: a magnetic hard disk; a floppy disk; an optical disk, such as a CD-ROM and a DVD; a magnetic tape; a flash removable memory; a memory chip; and/or other types of media that can store machine-readable instructions thereon. A "computer system" may refer to a system having one or more computers, where each computer may include a computer-readable medium embodying software to operate the computer. Examples of a computer system may include: a distributed computer system for processing information via computer systems linked by a network; two or more computer systems connected together via a network for transmitting and/or receiving information between the computer systems; and one or more apparatuses and/or one or more systems that may accept data, may process data in accordance with one or more stored software programs, may generate results, and typically may include input, output, storage, arithmetic, logic, and control units.
A "network" may refer to a number of computers and associated devices that may be connected by communication facilities. A network may involve permanent connections such as cables or temporary connections such as those made through telephone or other communication links. A network may further include hard-wired connections (e.g., coaxial cable, twisted pair, optical fiber, waveguides, etc.) and/or wireless connections (e.g., radio frequency waveforms, free-space optical waveforms, acoustic waveforms, etc.). Examples of a network may include: an internet, such as the Internet; an intranet; a local area network (LAN); a wide area network (WAN); and a combination of networks, such as an internet and an intranet. Exemplary networks may operate with any of a number of protocols, such as Internet protocol (IP), asynchronous transfer mode (ATM), and/or synchronous optical network (SONET), user datagram protocol (UDP), IEEE 802.x, etc. The invention involves an improved system, method and apparatus for better management of a premise using sensor and CCTV feeds and analytics. Moreover, the invention involves the use of an automated system employing video and sensor analytics for monitoring retail business processes. Video analytics may refer to the application of computer vision techniques to extract useful data or information from video streams or video sequences. Specifically, the invention may be applied for a number of purposes in a typical retail and commercial premise (e.g., shopping mall, grocery store, retail outlets, quick service restaurants, department store, banking, manufacturing units, hotels, market research government and other brick-and-mortar setups etc.).
The exemplary automated video surveillance system of the invention may combine information and video feed from multiple cameras. This can enable applications like cross-camera tracking of targets. Data from multiple cameras can be combined and all applications listed above applied to the combined information. This way the shopping habits of customers may be analysed in more details than just relying on a single camera.
The video feeds may be analysed in a real-time mode or the system may also operate in an off-line mode wherein queries may be applied to archived video feeds. In the off-line mode, the user may look for activities by mining the video feeds, instead of performing the entire video analytics again.
Examples of applying the invention for better management of a premise using sensor and CCTV feeds and analytics may include, for example:
- Measuring footfall of persons in a premise with uniform based staff segregation.
- Measuring groups entering a premise.
- Measuring Men/Women entering a premise.
- Measuring time spent by shoppers inside a premise.
- Measuring in-premise dwell times.
- Measuring in-premise shopper journeys: Aggregate and Individual shopper journeys.
- Measuring shopper percentages visiting sections of a premise.
- Capturing hand-movement and tracking hand motions.
- Mapping cash transactions, taking into account the time period from opening to receipt printing.
- Measuring staff productivity at manufacturing units, store and
restaurants.
- Measuring demo-time in a premise for example, a jewelry store. - Measuring how many trials are happening in a premise, for example in an apparel store.
- Smile-o-meter, measuring smiles of a shopper.
- Counting number of trolleys in a Hypermarket Billing Till.
- Measuring queue dispersions.
- Measuring table turns and table cleanliness in a premise for example, in a restaurant.
- Recognizing Shape, form, color and pattern.
- Measuring the path of a specific shopper without identifying the
shopper's demographics.
- Monitoring the presence of demo phones at the outlet for a mobile
manufacturer.
- Monitoring the uptime of demo laptops at the outlet for a laptop
manufacturer.
- Measuring number of people who enter a banquet hall in a hotel.
- Counting number of plates in a hotel buffet banquet.
Examples of applying the invention for better management of a premise using sensor and CCTV feeds and analytics to provide relevant, "actionable" insights, for example:
- Footfall : By Section & by Category ( as a % of total store footfall)
- Shopper Journey :- Shopper Flow through the Store
- Dwell (Section-wise & Category-wise)
- No. of Sections within the store visited by shoppers
- Avg. Shopping Time of Customer by time slots ( Weekend / Weekday)
- Weekly Trends : at store / City & Chain level
- Monthly Trends : City / Region / Chain level
- Quarterly Trends : City / Region / Chain / Account level No. of people in a banquet hall. Plate count in a buffet banquet. Repeat Customers and shoppers.
FIG 1 illustrates a block diagram providing an exemplary embodiment of the sensor 312 used in the invention for better management of a premise using video and sensor analytics. The sensor 312 comprises a processing unit 102 which is used to convert the video feed and the Media Access Control (MAC)-IDs of the Bluetooth and Wi-Fi enabled devices into metadata. The processing unit 102 is also responsible for submitting the metadata collected via video feed as well as the MAC-IDs to web server or cloud server using a network 100. The processing unit 102 checks for responses from the Wi-Fi unit and get the Wi-Fi MAC-IDs of the customer's handheld devices that have been detected by it in the vicinity. The processing unit 102 likewise checks for responses from Bluetooth unit and gets the Bluetooth MAC-IDs of the customer's handheld devices that have been detected by it in the vicinity. In addition, the processing unit 102 detects the network i.e. internet connectivity and thus submits the collected metadata to the web-server using store network i.e. internet. The processing unit 102 is also capable to save the collected and processed metadata for later usage and troubleshooting purposes in the in-built internal storage database. The sensor 312 further comprises a Wi-Fi unit 104 which is configured to detect the MAC-IDs of the Wi-Fi enabled handheld devices of the customers which are within the range of the sensor 312. The Wi-Fi unit 104 is connected to the processing unit 102. The sensor 31 2 is configured to detect any Wi-Fi enabled handheld device and it's unique MAC-ID. Even if the device is not connected to Wi-Fi, the sensor can track the unique MAC-ID of the handheld device. The sensor 312 further comprises a Bluetooth unit 106 which is configured to detect the MAC-IDs of the Bluetooth enabled handheld devices of the customers which are within the range of the sensor 312. The processing unit checks/scans for the connected and unconnected Bluetooth devices and wireless MAC-IDs of the customer's handheld devices in the vicinity using the Wi-Fi unit 104 and Bluetooth unit 106. The response from the Bluetooth 106 and Wi-Fi 104 units are checked by the processing unit 102. If the processing unit of the sensor does not get a response from the Bluetooth and Wi-Fi units then the Bluetooth and Wi-Fi units are restarted.
The sensor 312 is configured and connected to take video feed 108 from an existing CCTV infrastructure at the store. The CCTV infrastructure includes cameras, digital video recorder (DVR) connected to the network of the system. Being connected to the network 100 i.e. the internet, the processing unit 102 forwards the saved metadata on the sensor to the computing resources and cloud storage 1 10 for further processing to analyse and generate the analytics report for the client or retail store.
FIG. 2 illustrates an exemplary flow diagram showing an example process explaining how the hardware components within the sensor are interconnected and work together. The example process 200 starts by switching on the processing unit of the sensor i.e. device linkit is switched on. The sensor's activities can be divided into (i) MAC- ID related actions and (ii) Video analytics related actions for generating reports for client or retail store.
To start with MAC-ID related actions, the Bluetooth and Wi-Fi units are switched on at step 202. Thereafter, the response from the Bluetooth and Wi-Fi units are checked by the processing unit at step 204. If the processing unit of the sensor does not get a response from the Bluetooth and Wi-Fi units then the Bluetooth and Wi-Fi units are restarted at step 224. If the processing unit of the sensor gets a response from the Bluetooth and Wi-Fi units then at step 206 the processing unit checks/scans for the connected and unconnected Bluetooth devices and wireless MAC-IDs of the customers handheld devices in the vicinity. The metadata including the video fee and detected MAC-IDs are stored in the internal database which is inbuilt in the processing unit or which may include SD Card or other storage devices at step 226. Coming now to Video analytics related actions, the sensor fetch video feed from the CCTV camera at step 216. The CCTV infrastructure includes cameras, digital video recorder (DVR) or the like. One copy of the video feed is left on the DVR and one copy of the video feed is stored in the local storage at step 218. The video feed is processed by the sensor and converted into metadata at step 220. Once converted into metadata, the video footage may be deleted at step 222 and the converted metadata is stored in a database which may include SD Card or other storage devices. Verifying the internet connectivity of the processing unit at step 208, the saved metadata and the MAC-IDs are forwarded to the web server or cloud server at step 210. Once uploaded on the cloud server, the metadata and the MAC-IDs are stored, processed and analysed as explained in flow diagrams depicted in FIGS. 4 and 5. Thus, the system including the sensor removes all personal idenifiable information (Pll) and only uploads the metadata on a web server or cloud server thus maintaining complete privacy of the data and ensuring that no Pll from the video feed goes out of the retail store. In one embodiment, as in step 212, for internet connectivity, the Ethernet or Wi-Fi or GPRS is used following the order.
FIGS. 3 and 3A illustrates exemplary embodiments of a system 300 used for better management of a premise using video and sensor analytics. The sensor 312 may take video feed from an existing CCTV infrastructure at the store. The CCTV infrastructure includes cameras 300, digital video recorder (DVR) connected to a network of the system 310. A CCTV video camera 310 is positioned to view an area of a retail store to obtain video data/feeds. Optionally, the video data from the CCTV 300 may be stored in a video database. The system 31 0 ensures that video feeds never go out of the store premises and are processed locally securing complete privacy of the information. The sensor 312 is configured to detect any Wi-Fi or Bluetooth enabled handheld device 1 16 which are within the range of the sensor. Even if the device 1 16 is not connected to Wi- Fi or Bluetooth, the sensor 312 can track the unique media access control (MAC) ID of the handheld device 1 16 to generate trends as illustrated in FIG. 3 and forward curated, personalized content based on online behaviour, past purchases, opt-in provided, demographics etc. to the shopper's handheld device 1 16 as illustrated in FIG. 3A.
A network including a store internet switch 314 connects the sensor 312 with the store DVR so that the sensor 312 can access and process the stored video feeds in real-time. The sensor 312 fetches video feed from the CCTV 310 and leaves one copy of the video feed on the DVR and gets one copy to the local storage. The sensor 312 process the fetched video feed and convert the video feed into metadata. Metadata usually is "data" (information) that provides information about other data. Metadata summarizes basic information about data, which can make finding and working with particular instances of data easier. The copy of video feed which has been converted into metadata is deleted. The sensor 312 forwards the metadata to the webserver 318 which deployed with a web service which is connected to a database server 320 and in turn connected to a web dashboard 326 configured to process the metadata for generating analytics reports 324 for the client 326.
The database server 320 is the back-end system using client/server architecture that performs tasks such as data analysis, storage, data manipulation and archiving etc. to generate web dashboards that can be used by clients 326 or other business stakeholders.
FIG. 4 illustrates a flow diagram of how the sensor interacts with other components of the system depicted in FIGS. 3 & 3A. The example process 400 starts by switching on the sensor, the CCTV camera, DVR and the network, such as internet. The sensor is connected to the DVR via the network. The sensor accesses the video feed from the DVR and converts the video feed into metadata at step 402. The sensor also collects the MAC-IDs of the smartphones of the customers who are in vicinity at step 402. The sensor uses the network including the store switch at 404 and in turn forwards the metadata and the MAC-IDs to the web server or cloud server at step 406 using the store network. The sensor has storage capability to save the data in case the network is not available and is unable to forward the metadata to the web server or cloud server. The sensor is also configured to forward the data to the web server or cloud server once the network is available. A confirmatory determination is made at step 406 if the metadata has been forwarded to the web server or cloud server. After having the confirmation, at step 410, a web service deployed on the web server or cloud sever to which sensor submits the metadata performs proper verification to confirm that there is no data loss during the submission. After the successful verification, the sensor may delete the metadata from its inbuilt internal storage or other local storage. Thereafter, the data received by the web server is forwarded to the app server at step 412 wherein the app server combines the received metadata with the store configuration data, which was provided at the time of store setup and initial roll-out. At step 414, the combined data is further analysed and arranged with the help of business intelligence software in order to create the relevant analytics reports for the clients as per their need at step 41 6. The client is provided with the prepared report at step 418 which can be accessed on the web- dashboard.
The app server at step 420 also determines and prepares the personalized content that needs to be provided/shown to the specific consumers on the basis of selected MAC- ID. The personalized content prepared by the app server is provided to the customer to the web server over a web-dashboard at step 422. Web server renders the personalized content on the customers handheld device which can be accessed by the customer at step 424. The personalized content may be accessed by the customer while he /she is still within the range of the sensor.
FIG 5 illustrates a flow diagram of an example process explaining the customer engagement opportunities in terms of receiving personalized messages and their feedback to the brands, shopping experience or on other factors. The example process 500 starts by switching on the sensor, the CCTV camera, DVR and the store network, such as internet. The sensor is connected to the DVR via the store network. The sensor accesses the video feed from the DVR and converts the video feed into metadata. The sensor also detects the MAC-IDs of the smartphones of the customers who are in vicinity at step 502. The metadata and the detected MAC-IDs are sent to the web-server at step 504 using the store network which includes internet or GSM network or any other suitable network. The web-server received the MAC-IDs and forwards them to the database server at step 506. Thereafter, the database server applies the logic of personalized content specific to the identified customer based on their MAC-ID at step 508 to generate a personalized message for the targeted/identified customer. The generated personalized data or message is sent back to the web-server at step 510 by the database server. The web-server in turn renders the personalized data or message onto the web browser at step 512 which can be accessed by the customer on their handheld device at step 514. The reports of personalized data or messages sent to the identified customers are also made available to the client or stakeholders on the web dashboard at step 518. The client or retail store or stakeholder may monitor the reports and also make changes in the offers or other information in database server as per their need at step 520. The example process ends at step 516.
Referring now to FIG 6 which illustrates a flow diagram of an example process explaining - how retailer stores can establish the benchmark conversion for their store(s). The sensor 312 delivers Store Opportunity Scorecard for retailers - including small format, large format and chain stores, which tells them their true conversion potential, helping them set the right benchmarks. This is done by calculating parameters such as drop-ins, drop-offs, browsers, defectors, and repeats. The sensor 312 combines the VA and Wi-Fi intelligence to qualify the vanilla footfall numbers and help retailers establish a fact-based conversion benchmarks. As illustrated in FIG 6, the sensor 312 takes the footfall from Video feeds, maps them to MAC IDs to qualify the footfall. The example process 600 starts by switching on the sensor, the CCTV camera, DVR and the store network, such as internet. The sensor is connected to the DVR via the store network. The sensor accesses the video feed from the DVR and processes the video feed and/or converts the video feed into metadata at step 602. The sensor also detects the MAC-IDs of the smartphones of the customers who are in vicinity at step 604 and accordingly calculates the footfall to the store. Thereafter, at step 606 using the Wi-Fi data the footfall count are qualified into time spent by the customer inside the store. It is determined at step 608, for example, percentage of customers or shoppers who have spent more than twenty minutes inside the store end up buying a product. Thus it is determined that a twenty minute period spent by the customer within the store must be at least he minimum conversion potential that the store should have which helps in increasing the business. Taking into account the foregoing, the sensor 312 benchmarks the conversion potential that the retail store must have at step 610 and prepare a report thereon. Such benchmark reports are published at step 612 and made available to the client or retail store at step 614.
FIG 7 illustrates a flow diagram of an example process explaining - how retail stores can build a two-way customer engagement platform using proximity marketing and customer feedback. The flow diagram describes how brands and customers can connect with each other using the sensor. Free W-Fi is a catching up as a latest trend in the retail landscape to drive in more traffic. The Sensor offers the retailer stores a solution of converting their Wi-Fi into a hotspot and offers it as Free Wi-Fi for shoppers/customers. This gives brands or retail store a dual advantage of driving traffic into store(s), and also get a platform to send marketing content or promotions to them just in time when they are in or around the stores. It also acts as a unique two way engagement channel, where shoppers can give real-time feedback to the brand on their shopping experience, issues or high points.
The example process 700 starts by switching on the sensor, the CCTV camera, DVR and the store network, such as internet deployed at the retail store as in step 702. The sensor extends the store network with a capability for customers/shoppers to latch on to it at step 704. At step 706, the customer can see the store's Wi-F network on his/her handheld device. If the customer wishes, he/she can latch onto the store network by agreeing to the terms and conditions as in step 708. Once the connection is established, the customer is digitally connected to the retail store or brand as in step 710 while being in the store as in step 716. Being connected, the store or the brand can send eCoupons, vouchers, personalized content, shopping related information, information relating to discounts and offers to the customer at step 712. The customer can also provide his/her feedback or comments to the brands or about the in-store experience and about any merchandise that the customer may not have found in-store or comment of the products, provide complaints, suggestions to the retail store or brand as in step 714.
FIG 8 is a flow diagram of an example process explaining - how Quick Serve Restaurants can reconcile transactions to minimize billing errors or frauds. The sensor can help Quick Serve Restaurants to reconcile transactions. The example process 800 starts by switching on the sensor, the CCTV camera, DVR and the store network, such as internet deployed at the retail store as in step 802. The sensor processes video feeds from the CCTV cameras as in step 806 installed at the billing counter as in step 804. The sensor runs the built-in checks and algorithms on the video feeds and raises alerts when any of the pre-specified conditions are not met as in step 810. These conditions include customer hand movement, till opening, and receipt printing as in step 808. For example, if for any transaction, only till opening or/and receipt printing is detected but no hand movement is seen, then an alert is raised as in step 810. The store manager can then track and question that transaction. Thus, the client or retail store is able to monitor each transaction and ensure that it is duly entered in the billing system as in step 812. The sensor is also able to reconcile all the transactions as in step 814. The example process ends at step 816.
The process of FIGS. 2, 4, 5 to 8 and each of the other processes discussed herein may be implemented in hardware, software, or a combination thereof. In the context of software, the described operations represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, perform the recited operations. Generally, computer- executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. From the foregoing description, it will be apparent to one ordinarily skilled in the art that many changes and modification can be made thereto without departing from the spirit or scope of the invention as set forth herein. Accordingly, it is not intended that the scope of the foregoing description be limited to the description set forth above, but rather that such description be construed as encompassing all of the features of patentable novelty that reside in the present invention, including all the features and embodiments that would be treated as equivalents thereof by those skilled in the relevant art.