US20170189803A1 - Task-oriented feedback using a modular sensing device - Google Patents

Abstract

A wearable device can be worn by a user, and can include a mode selector, one or more sensors, a signal detector to detect wireless signals, a feedback mechanism comprising at least one of a haptic system, an audio system, or a visual system and providing feedback outputs. The wearable device can also include a controller that can initiate a selected mode from a plurality of modes on the wearable device, where the selected mode causing the controller to execute a real-world game. In accordance with the selected mode, the controller can interpret sensor data from the one or more sensors and inputs from the signal detector, where the sensor data corresponds to a series of actions performed by the user utilizing the wearable device. Based the interpretations, the controller can generate feedback responses via a feedback mechanism.

Description

CROSS REFERENCE TO RELATED APPLICATIONS
• This application claims the benefit of priority to U.S. Provisional Patent Application Ser. No. 62/274,514, entitled “PORTABLE SENSING DEVICE FOR PROCESSING GESTURES AS INPUT,” and filed on Jan. 4, 2016; and U.S. Provisional Application Ser. No. 62/346,216, entitled “MODULAR SENSING DEVICE FOR PROCESSING GESTURES AS INPUT,” and filed on Jun. 6, 2016; the aforementioned priority applications being hereby incorporated by reference in their respective entireties.
BACKGROUND
• Wearable device technology in consumer electronics is rapidly being integrated into routine user activities, such as sporting activities, content viewing or browsing, and task-oriented activities (e.g., gaming). Furthermore, wireless networks typically utilize protocols that enable wireless devices to detect signal sources from other devices for initiating data and communication links. Such networks are typically implemented using networking hardware, which may be incorporated in various wireless network devices, such as access points (APs), peer-to-peer (P2P) devices, wireless local area network (LAN) equipped devices, and the like—each advertising a unique identity (e.g., a media access control (MAC) address) indiscriminately to devices within range. Connections may be established with such devices to transmit and receive data.
BRIEF DESCRIPTION OF THE DRAWINGS
• The disclosure herein is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements, and in which:
• FIG. 1A illustrates a block diagram of an example modular sensing device for use in controlling a remotely operated device, according to examples described herein;
• FIG. 1B is a block diagram illustrating an example modular sensing device, as shown and described herein;
• FIG. 2 is a flow chart describing an example method of generating commands by a modular sensing device, in accordance with examples described herein;
• FIG. 3 illustrates an example modular sensing device insertable into a plurality of compatible apparatuses;
• FIG. 4 is a block diagram illustrating an example modular sensing device for acquiring virtual or digital resources;
• FIG. 5 is a block diagram illustrating an example virtual resource management system utilized by a wearable device;
• FIG. 6 is a flow chart describing an example method of acquiring virtual resources using a wearable device;
• FIGS. 7A and 7B are flow charts describing example methods of acquiring and utilizing virtual resources in connection with gameplay;
• FIGS. 8A, 8B, and 8C illustrate unique identifier logs and association tables utilized in connection with virtual resource association, acquisition, and allocation;
• FIG. 9 illustrates a wearable device pairing triggering virtual resource data sharing in connection with gameplay;
• FIGS. 10A and 10B illustrate a wearable device pairing triggering an interactive mode between a pair of proximate users;
• FIG. 11 is a flow chart describing an example method of implementing an interactive mode between a pair of user utilizing a corresponding pair of wearable devices;
• FIG. 12 is a flow chart describing an example method of initiating a training mode on a wearable device in connection with a self-propelled device;
• FIG. 13 is a flow chart describing an example method of implementing a wearable device in a sword mode;
• FIG. 14 is a block diagram of an example computer system upon which examples described herein may be implemented;
• FIG. 15 is a block diagram of a mobile computing device upon which examples described herein may be implemented;
• FIG. 16 is a block diagram of an example portable sensing device upon which examples described herein may be implemented;
• FIG. 17 illustrates an embodiment of multiple sensing devices that concurrently provide input for a program or application which utilizes the inputs, along with inferences which can be made about a person or object that carries the devices, according to one or more examples;
• FIG. 18 illustrates a system which concurrently utilizes input from multiple modular sensing devices in connection with execution of an application or program;
• FIG. 19 illustrates an example of a modular sensing device insertable into a wrist worn apparatus; and
• FIG. 20 illustrates an implementation of the modularized sensing device, in accordance with examples described herein.
DETAILED DESCRIPTION
• Examples described herein relate to a multi-modal modular sensing device, worn or carried by a user (e.g., as a wrist-worn device), to enable a variety of interactions with other devices through sense movement of the modular sensing device. Among other activities, examples provide for a modular sensing device that can individually, or in combination with another device (e.g., controller device, such as a mobile computing device) control other devices, interact with compatible devices of other users, and/or operate in connection with task-oriented activities (e.g., gameplay). In some examples, the modular sensing device corresponds to a wearable device (e.g., a watch, a pendant for a necklace, a hat, glasses) can be placed in multiple modes to, for example, control the characteristics of movement of another device, or interact with the real-word. For example, the modular sensing device can control acceleration and maneuvering of a self-propelled device.
• In certain aspects, a portable modular sensing device can include an inertial measurement unit (IMU), and can be operated in multiple modes in which gestures (e.g., arm gestures) may be translated based on the particular mode of the wearable device. In one example, the modular device can be insertable into multiple devices, such as a wrist worn device, clothing, an accessory device (e.g., a toy sabre or sword), a wearable pendant, ring, hat, and the like. The modular sensing device can further include output devices, such as lighting elements (e.g., one or more LEDs), an audio device (e.g., a speaker), and/or a haptic feedback system. In one example, the modular sensing device further includes a charging port (e.g., a mini-universal serial bus port) to enable charging of a power source included in the device.
• According to some examples, a modular sensing device is operable to detect its own movement in three-dimensional space, using the IMU. In some implementations, the IMU can be an integrated device. Alternatively, the IMU can be implemented through a combination of sensors, such as a three-dimensional accelerometer or gyroscope. In certain examples, the modular sensing device can include a processor and memory to interpret the sensor data, and to communicate interpreted sensor data to another device (e.g., mobile computing device) using a wireless connection (e.g., BLUETOOTH). In variations, the sensor data can generate data that is either raw data or processed, depending on the processing resources that are selected for the portable device, and the processing load which is implemented for the portable device.
• In many implementations, the modular sensing device can generate output (e.g., haptic, audio, and/or visual output) or control commands for operating a remote controlled device by utilizing state machines in memory that correspond to individual gestures. Depending on a selected mode of the modular sensing device, processing resources of the modular sensing device can monitor logical state machines or automatons that correspond to specified sensor data combinations (e.g., based on gestures performed by a user). For example, a single state machine can correspond to an initial arm raising gesture performed by the user wearing a wrist-worn device having the modular sensing device. A sensor profile from a single sensor or multiple sensors included in the IMU (e.g., gyroscopic sensor and/or accelerometer) can indicate the initial arm raising gesture, which can trigger a state transition for the state machine. This state transition can cause processing resources of the modular sensing device to automatically generate a control command to cause the remote controlled device to accelerate.
• Additionally, a second state machine can correspond to an increased angle in the arm raising gesture. When the sensor profile of the IMU indicates the gesture above a certain predetermined angle, a state transition can be triggered for the second state machine, which can trigger an increased acceleration command to be generated automatically. Several state machines can be comprised in memory of the modular sensing device, and can each correlate to a specified sensor data profile, from simple single-action gestures, to complex multiple-motion gestures. Furthermore, an interpretation engine (e.g., a processor) can interpret each state transition for each state machine based on a particular mode and/or sub-mode of the modular sensing device.
• As used herein, a “modular sensing device” can comprise an electronic device that includes sensor resources for detecting its own movement, and of dimension and form factor suitable for being insertable into any number of devices configured for receiving the device. Numerous examples of modular sensing devices are provided in the context of a “wearable device,” such as a wrist worn device (e.g., watch, watch band, bracelet). But as noted by other examples, variations to the type and form factor of a wearable device can vary significantly, encompassing, for example, eyeglasses, hats, pendants, armbands, glasses and various other form factors. While many examples describe functionality in the context of a wearable device, embodiments extend such examples to other form factors in which a modular sensing device may be inserted, such as wands, fobs, wielded toys, or mobile communication devices.
• In many examples, the wearable device can include one or more sensors to detect the device's own movements. In particular, a wearable device includes an accelerometer and/or a gyroscopic sensor. In some examples, sensor data, corresponding to user gestures performed by the user wearing the wearable device, can be translated into control commands or data packets to be transmitted and implemented based on the selected mode of the wearable device. According to many examples, the wearable device can include an inductive interface to inductively pair with other devices, which can trigger a specified mode on the wearable device. For example, an inductive pairing between the wearable device and a self-propelled device can trigger a drive mode in which the wearable device can be utilized by the user to operate the self-propelled device. Additionally or alternatively, the wearable device can include an input mechanism, such as an analog or digital button, that enables the user to select a particular mode and/or scroll through a series of modes for the wearable device.
• Among other functionality, some examples described herein provide for alternative modes of operation, including, for example (i) a “drive mode” in which the wearable device is utilized to control a self-propelled device; (ii) a “control mode” in which the wearable device is utilized in connection with smart home devices; (iii) a “finding mode” or “finder mode” in which the wearable device is utilized to detect virtual or digital resources; (iv) a “mining mode” which can be initiated by the user to collect virtual resources when they are detected in the finder mode; (v) a “training mode” in which the wearable device is utilized in connection with a self-propelled device to assist the user in training for certain achievements or for increasing the user's abilities to perform task-oriented activities (e.g., increasing skills for a subsequent game or sporting activity); (vi) a “sword mode” in which the wearable device provides feedback (e.g., haptic, audio, and/or visual feedback) when the user performs actions while holding an object; (vii) a “default mode” in which the device monitors for and detects other proximate wearable devices (e.g., wearable devices of the same type) which enables the users to pair with each other's wearable devices; (viii) a “interactive mode” or “battle mode” selectable in response to two or more device pairings in which users are able to interact with each other with predetermined sets of actions (e.g., offensive and defensive actions learned and perfected by users practicing in the training mode); (ix) a “sharing mode” selectable in response to two or more device pairings in which users can share information stored in each other's wearable devices, or user accounts associated with the wearable devices (e.g., sharing collected virtual resources discovered and mined in the finder and mining modes to be expended or consumed in a gameplay environment); and (x) a “gaming mode” in which the wearable device can be utilized in connection with a game.
• Still further, numerous examples make reference to a “self-propelled” device. A self-propelled device can include, for example, a device that can be wirelessly and remotely controlled in its movement, whether the movement is on ground, water, or air. For example, a self-propelled device can include a wirelessly and remotely controlled drone, car, airplane, helicopter, boat, etc. While conventional examples enable control of a self-propelled device, conventional approaches generally utilize a perspective of the device being controlled. While some conventional devices, for example, enable a computing device held by the user to project a perspective of the device under control, examples described herein enable control of such devices to utilize an orientation of the user. Specifically, some examples include a modular sensing device that can determine an orientation of the user, and further enable control of the self-propelled device through an environment that accommodates or is in the perspective of the user, based on the orientation of the user (as determined by the modular sensing device). By way of example, the control of a self-propelled device can be projected through an orientation or perspective of the user for purpose of a virtual environment.
• Some examples include a modular sensing device having a wireless communication module (e.g., a BLUETOOTH low energy module) that enables communication of sensor data (e.g., raw sensor data from the accelerometer or gyroscopic sensor), or translated data (i.e., translations of the sensor data based on the selected mode of the wearable device). In some examples, the sensor data may be relayed for translation by a mobile computing device before being transmitted to another device (e.g., a paired wearable device or a paired self-propelled device). In other examples, processing resources of the wearable device can execute mode instructions, based on the selected mode, to translate the sensor data (e.g., via use of state machines that trigger control commands or feedback based on a selected mode of the modular sensing device) for direct transmission to one or more other devices, as described herein.
• As used herein, “body part gestures” or “user gestures” include gestures performed by a user while utilizing the wearable device. For example, the wearable device may be a wrist-worn device, in which case the body part or user gestures may comprise arm gesture, and can include any number of physical movements or actions that affect the sensors of the wearable device when it is worn on the wrist. Such movements and actions can include shaking, arm movements (e.g., raising, lowering, pointing, twisting, and any combination thereof), wrist movements, hand actions (such as grasping or grabbing), and the like. However, the wearable device is not limited to wrist-worn devices, but may be utilized as a ring (e.g., a finger-worn device), an ankle-worn device, a neck-worn device, a head-worn device, a belt (e.g., a waist-worn device), etc. Thus, user gestures performed using the wearable device can be any actions or movements in which correlated sensor data from sensors of the device can be translated into commands, instructions, feedback, etc. depending on the mode of the wearable device.
• Among other benefits, examples described herein achieve a technical effect of enhancing user interactivity with other devices and other users. Such interactivity may include utilizing the modular sensing device to control a self-propelled device, interact with other users of wearable devices, collect and share data, control smart home devices, interact with the real-world via a gaming interface through the modular sensing device, and the like.
• One or more examples described herein provide that methods, techniques, and actions performed by a computing device, a modular sensing device, or a self-propelled device are performed programmatically, or as a computer-implemented method. Programmatically, as used herein, means through the use of code or computer-executable instructions. These instructions can be stored in one or more memory resources of the computing device. A programmatically performed step may or may not be automatic.
• One or more examples described herein can be implemented using programmatic modules or components of a system. A programmatic module or component can include a program, a sub-routine, a portion of a program, or a software component or a hardware component capable of performing one or more stated tasks or functions. As used herein, a module or component can exist on a hardware component independently of other modules or components. Alternatively, a module or component can be a shared element or process of other modules, programs or machines.
• Some examples described herein can generally require the use of computing devices, including processing and memory resources. For example, one or more examples described herein can be implemented, in whole or in part, on computing devices such as digital cameras, digital camcorders, desktop computers, cellular or smart phones, personal digital assistants (PDAs), virtual reality (VR), augmented reality (AR), or mixed reality (MR) headsets, laptop computers, printers, digital picture frames, and tablet devices. Memory, processing, and network resources may all be used in connection with the establishment, use, or performance of any example described herein (including with the performance of any method or with the implementation of any system).
• Furthermore, one or more examples described herein may be implemented through the use of instructions that are executable by one or more processors. These instructions may be carried on a computer-readable medium. Machines shown or described with figures below provide examples of processing resources and computer-readable mediums on which instructions for implementing examples can be carried and/or executed. In particular, the numerous machines shown with examples include processor(s) and various forms of memory for holding data and instructions. Examples of computer-readable mediums include permanent memory storage devices, such as hard drives on personal computers or servers. Other examples of computer storage mediums include portable storage units, such as CD or DVD units, flash memory (such as carried on smart phones, multifunctional devices or tablets), and magnetic memory. Computers, terminals, network enabled devices (e.g., mobile devices, such as cell phones) are all examples of machines and devices that utilize processors, memory, and instructions stored on computer-readable mediums. Additionally, examples may be implemented in the form of computer-programs, or a non-transitory computer usable carrier medium capable of carrying such a program.
• System Description
• FIG. 1A illustrates a block diagram of an example modular sensing device for use in controlling a remotely operated device, according to examples described herein. Themodular sensing device100 shown inFIG. 1A may be integrated with a wearable device102 (e.g., a wrist-worn device, pendant, clothing, a hat, eyeglasses, an ankle-worn device, a ring, etc.), or can be inserted into a compartment or otherwise included as a part of thewearable device102. Thewearable device102, ormodular sensing device100 disposed therein, can include amode selector110, such as an analog button that enables the user to select a mode of the device. In one example, repeateduser input111 on themode selector110 enables the user to scroll through a list of available modes. Such modes include, but are not limited to a drive mode, a control mode, a finder mode, a mining mode, a training mode, a sword mode, a default mode, an interactive or battle mode, a sharing mode, and a gaming mode. As shown in the block diagram, auser input111 on themode selector110 can cause aprocessor120 of themodular sensing device100 to generate anoutput132 that indicates the particular mode selected. For example, themodular sensing device100 can include a number ofoutput devices130, such as an audio device, a haptic system, and/or visual elements (e.g., an LED array or display). Eachuser input111 on themode selector110 can trigger an audio, haptic, and/orvisual output132, indicating the particular mode (e.g., a drive mode). As an example, theoutput132 can comprise a voice output that states the selected mode, or a combination of voice, visual, and haptic output.
• Additionally or alternatively, the user may connect thewearable device102 with a mobile computing device, such as the user's smart phone or tablet computer. Mode selection may be performed automatically by the user initiating a designated application of the mobile computing device, such as a smart home application, a controller application (e.g., to control a self-propelled device), a gaming application, and the like. In variations, the user can execute a designated application in connection with thewearable device102 that enables the user to scroll through the various modes. The user may scroll through the modes on the mobile computing device, or via successive selection inputs on themode selector110, which can trigger the mobile computing device to display a selectable mode. In other variations, multiple types of inputs can be performed in themode selector110, such as tap gestures and tap and hold gestures, which can correlate to scrolling through the modes and selecting a particular mode respectively. As provided herein, themode selector110 can be an input mechanism such as an analog or digital button, a touch panel such as a track pad, or a miniature touch-sensitive display device.
• According to examples provided herein, themodular sensing device100 can include amemory115 that stores mode instruction sets executable by theprocessor120 based on the mode selected by the user. Each mode instruction set can cause theprocessor120 to interpretsensor data137 in a particular manner. Along these lines, same orsimilar gestures106 performed by the user can result in different generatedoutputs132 or commands108 depending on the mode selected by the user. In some implementations, selection of a particular mode can cause theprocessor120 to initiate a communications module125 of themodular sensing device100 to establish a wireless connection with another device. In one example, themodular sensing device100 can include a BLUETOOTH low energy module to enable communications with one or more other devices, such as a second modular sensing device or a remotely operateddevice140.
• Themodular sensing device100 can be relatively small in size compared to current computing devices, and in many implementations, themodular sensing device100 does not include a power-intensive memory and computational resources. In such implementations, the memory and/or memory controller can be comprised of or implement a series of state machines that, when transitioned, can trigger a particular output automatically. Further description of the state machine implementations is provided below with respect toFIG. 1B. Furthermore, examples described herein with respect to the drive mode of themodular sensing device100 can also be implemented in the state machine examples described herein.
• As provided herein, thememory115 can include a drivemode instruction set117 executable by theprocessor120 in response to auser input111 on themode selector110. In some aspects, execution of thedrive mode instructions117 can cause theprocessor120 to initiate the communications module125 to establish a communications link104 with a proximate remotely operateddevice140. In variations, themodular sensing device100 can include aninduction interface127 which can trigger thecommunication link104 between themodular sensing device100 and the remotely operateddevice140. For example, upon selecting the drive mode, the user can place thewearable device102 within inductive range (e.g., ˜2-5 cm) of the remotely operateddevice140, which can include a corresponding induction interface. The inductive transfer of communication information can enable themodular sensing device100 to establish the communication link accordingly.
• According to examples described, themodular sensing device100 can further include an inertial measurement unit (IMU)135 that can comprise a gyroscope and an accelerometer for accurate measurement of the modular sensing device's100 movement and orientation. In variations, theIMU135 can further include a magnetometer to, for example, assist in calibration based on the orientation. Once thecommunication link104 is established in the drive mode, theprocessor120 can monitor thesensor data137 for theparticular gestures106 being performed by the user. In several examples, thegestures106 can correspond to the user raising or lowering an arm, and/or performing additional arm motions. Thesensor data137 from theIMU135 can comprise movement, position, and/or orientation information that theprocessor120 can interpret in accordance with the drive mode. For example, gestures106 performed by the user can be detected by theprocessor120 viasensor data137 from theIMU135. Each of thegestures106 can be interpreted by theprocessor120 as one or more control commands108 to be executed by the remotely operateddevice140.
• In one example, the drive mode can be automatically initiated in response to a particular detected gesture, regardless of the current mode of themodular sensing device100. This gesture can correspond to a distinct sensor data signature that, when detected by theprocessor120 in executing any mode, overrides that mode and initiates the drive mode automatically. Thus, upon detection of the distinct sensor data signature, theprocessor120 can automatically initiate the communications module125, establish the communications link104 with the remotely operateddevice140, and generate control commands108 based on the detectedgestures106 performed by the user in thesensor data137. Themodular sensing device100 may then dynamically transmit such control commands108 to the remotely operateddevice108 for execution as they are generated by theprocessor120. In one example, the specific gesture corresponds to a pushing motion with an arm wearing thewearable device102 performed by the user. As provided herein, this pushing motion can correspond to a specified sensor data signature not used for any other mode, and is therefore distinct to enable theprocessor120 to identify it irrespective of the current mode of themodular sensing device100.
• In certain examples, gestures106 such as raising an arm can cause theprocessor120 to generate acceleration commands to drive away from the user. Lowering the arm can cause the processor to generate deceleration and/or reverse commands. Further, moving the arm from side to side can cause theprocessor120 to generate steering or directional commands. For example, moving the arm left can cause the remotely operateddevice140 to turn left, and moving the arm right can cause thedevice140 to turn right as thedevice140 travels away from the user. Such control commands108 may be processed by a controller of the remotely operateddevice140, or may be directly executed on the drive motors of thedevice140 in order to accelerate and maneuver thedevice140 in accordance with thegestures106 performed by the user.
• Furthermore, in the drive mode, angular thresholds can be established in thedrive mode instructions117 that can determine the manner in which theprocessor120 interprets thesensor data137. When such thresholds are crossed, theprocessor120 can alter interpretation of thesensor data137 into alternative commands108. For example, as the user raises the arm above an angular threshold (e.g.,45 degrees), and/or changes an orientation of the arm (e.g., palm down to palm up), theprocessor120 can alter the interpretation of thesensor data137 such that remotely operateddevice140 drives towards the user as the arm is raised. Furthermore, in such a state, the directional interpretation of thesensor data137 can be reversed such that moving the arm left can cause the remotely operateddevice140 to turn right, and moving the arm right can cause the remotely operateddevice140 to turn left. This directional reversal triggered by the angular threshold, and in combination with the change in orientation of the user's arm, can create a palm control illusion of the remotely operateddevice140 by the user. Thus, in the drive mode, specified gestures detected in the sensor data137 (e.g., the user's arm rotating or crossing an angular threshold) can trigger theprocessor120 to interpret thesensor data137 differently, or inversely from an initial interpretation, in order to produce the illusion.
• FIG. 1B is a block diagram illustrating an example modular sensing device, as shown and described herein. As provided herein, themodular sensing device150 can be space limited, and can only include a limited amount memory and computational resources. In such implementations, themodular sensing device150 can represent each possible gesture that can be performed by a user as a state machine. Thus, for each gesture, a state machine corresponding to that gesture can either positively identify its gesture or negatively identify its gesture. When a positive gesture is identified, the state machine triggers a state transition which can cause anoutput generator160 to generate a particular output accordingly. As provided herein, theoutput generator160 andmemory180 can comprise a hardware controller implementing the various state machines to generate the output commands164.
• As an illustration, themodular sensing device150 can include amemory180 implementing a number of state machines (e.g.,SM1181,SM2183,SM3185,SM4187, . . . , SMN189), each being associated with a particular gesture. For example,SM1181 can be associated with the user raising an arm,SM2183 can be associated with the user lowering an arm,SM3185 can be associated with the user pointing an arm to the right, andSM4187 can be associated with the user pointing an arm to the left. Furthermore, any number of state machines may be implemented in thememory210 representing any number of gestures. At any given time step, the state machines can be instantiated for each gesture type, and each state machine can continuously inspect theinstantaneous sensor data167 from theIMU165 in order to initialize or instantiate, transition individual states along a state string, terminate a current state string, or trigger an accept state or final state. When the final state is triggered, this means that the particular gesture corresponding to that state machine has been performed by the user.
• According to examples, each state machine can consist of a finite set of states (a fixed string of states), one or more inputs, one or more outputs, a transition function, and an initial state. The state machine can be linked to a particular gesture by way of a sensor data signature, which can comprise an accelerometer data signature, gyroscope data signature, or a combination of both. Furthermore, the state machine can be linear and directional, with each state having a particular error tolerance in its sensor data signature. A final state of a state machine can thus be triggered when the full sensor data signature of a particular gesture is matched to thesensor data167 generated by theIMU165.
• In some aspects, if at any time after instantiating, an associated gesture corresponding to a respective state machine is not being performed, the input string for that state machine, and for that particular instantiation, automatically terminates. At any given instance, the state machine either terminates from the outset (e.g., an initial aspect of the sensor data signature for the gesture is not matched), or instantiates the input string towards the final state. At any given state along the input string, the state machine can terminate if the gesture being performed diverges from the error tolerances built into the state machine. If, however, each state along the input string is transitioned accordingly (i.e., thesensor data167 from theIMU165 matches the sensor data signature for that state machine within its error tolerances), the final state is triggered for that state machine.
• Thememory180 can include astate machine reporter182 that can report such final state transitions188 to anoutput generator160 of themodular sensing device150. Theoutput generator160 can be configured based on the particular mode of themodular sensing device150. Accordingly, final state reports188 from individual state machines can be interpreted differently, or can cause a particular output, depending on the mode. In other words, the output from themodular sensing device150 for a particular gesture (e.g., a backhanded swinging motion) can be different depending on the mode initiated via themode selector155. Furthermore, certain final state reports188 from thestate machine reporter182 can correspond tosub-mode triggers186 for the particular mode. Suchsub-mode triggers186 may not trigger an output, but rather trigger theoutput generator160 to alter interpretation of certain final state reports188 in order to generate an alternative output.
• Such outputs generated by theoutput generator160 can comprise control commands162 to operate a remotely operated device, such as acceleration, steering, and deceleration commands. In some aspects, theoutput generator160 can generate output commands164 for the modular sensing device's150haptic system192,visual system194, and/oraudio system196. Thus, as final state reports188 are received, theoutput generator160 can cause the haptic192, visual194, andaudio196 systems of themodular sensing device150 to produce individual or combined outputs. As described herein, such outputs can include vibration feedback or guidance, colored lights or display output, tonal outputs such as audible chimes that indicate positive and/or negative feedback, and voice output.
• In the example of the drive mode described with respect toFIG. 1A, each completed arm gesture can correspond to a finalstate machine report188, and can cause theoutput generator160 to generate acontrol command162 accordingly. In the example shown inFIG. 1B, the user raising an arm above a certain angle (e.g., five degrees) can cause a state machine corresponding to that gesture to transition to a final state. Thestate machine reporter182 can report thefinal state188 of that state machine to theoutput generator160, which, based on the drive mode interpretation, can generate acontrol command162 to initiate acceleration of the remotely operated device. Thiscontrol command162 may then be transmitted to the remotely operated device via thecommunications module175 of themodular sensing device150.
• Referring to bothFIGS. 1A and 1B, in certain modes, thecommunications module125,175 can act as a signal or beacon sensor which can provide signal data to theprocessor120 oroutput generator160. Based on the mode, theprocessor120 oroutput generator160 can generate a specified response based on the signal data. For example, the signal data can indicate a unique identifier of the signal source. Thememory115,180 can include a UID log of specified signal sources that allow the user to interact with such sources based on the mode of themodular sensing device100,150. For example, if the signal source corresponds to another modular sensing device, the user can be enabled to select a mode in which the user can interact with the other user (e.g., play a physical game involving gestures).
• As described with respect toFIG. 1A, the interpretation of thesensor data137 can be altered based on a particular gesture trigger, such as the user's arm exceeding an angular threshold. The analog with respect toFIG. 1B comprises a state machine in thememory180 that correlates to the particular threshold gesture. Thus, when thesensor data167 indicates the specified threshold gesture, the correlated state machine can transition to its final state, which can be reported to theoutput generator160 accordingly. Such afinal state report188 can comprise asub-mode trigger186, in that thesub-mode trigger186 causes theoutput generator160 to alter interpretation of certain final state reports188 while remaining in the same mode. In the drive mode example, asub-mode trigger186 can correspond to the user raising an arm above a certain threshold (e.g.,45 degrees), which can cause theoutput generator160 to, for example, reverse the interpretation of the final state reports188 corresponding to the directional commands for the remotely operated device.
• Additionally, each mode of themodular sensing device150 can include any number ofsub-mode triggers186 that cause theoutput generator160 to alter an interpretation of or disregard a particularfinal state report188 corresponding to a specified sensor data signature. For example, the drive mode can include a set of angular gesture thresholds (e.g., raising an arm beyond 45 degrees, lower the arm below 45 degrees, turning the arm from palm down to palm up). A state machine can be dedicated—within the specified mode—to a sensor data signature indicating the user gesture crossing a gesture threshold. Thus, when the user gesture crosses the gesture threshold, the dedicated state machine can transition to its final state, which, when reported to the output generator160 (i.e., as a sub-mode trigger186), can cause theoutput generator160 to alter interpretation of certain final state reports188 within that mode. In some aspects, disregarded final state reports prior to the sub-mode trigger can now trigger a specified output (e.g., an audio, haptic, and/or visual output, or a particular control command162).
• In further examples, when themodular sensing device150 is in a particular mode, a specific complex gesture—represented by a particular state machine in thememory180—can cause theoutput generator160 to reconfigure its interpretation of certain final state reports188, execute a sub-mode, or automatically initialize a different mode for themodular sensing device150. Accordingly, for any given instance,sensor data167 from theIMU165 can continuously cause the various state machines to instantiate, terminate, instigate state transitions, and/or transition to a final state. In aspects described, only when a state machine transitions to its final state does theoutput generator160 generate output commands164 and/or control commands162 to provide feedback, operative control over a remotely operated device, guidance via the output devices, and/or task-based instructions (e.g., in accordance with a particular game).
• Furthermore, the various aspects performed by themodular sensing device100 described with respect toFIG. 1A may also be performed by examplemodular sensing devices150 as shown and described with respect toFIG. 1B. Thus, the execution of designated modal instruction sets by themodular sensing device100 ofFIG. 1A—in which theprocessor120 directly interprets thesensor data137 based on the executing instruction set—may be substituted by the state machine examples as described with respect toFIG. 1B. In other words, the limited memory and computational resources of themodular sensing device150 ofFIG. 1B may be compensated by attributing sensor data signatures to state machines, requiring less memory and processing without sacrificing functionality.
• Methodology
• FIG. 2 is a flow chart describing an example method of generating output commands by a modular sensing device, in accordance with examples described herein. In the examples described with respect toFIG. 2, reference may be made to reference characters representing like features as shown and described with respect toFIGS. 1A and 1B. Furthermore, the methods and processes described with respect toFIG. 2 may be performed by an examplemodular sensing device100,150 as shown and described with respect toFIGS. 1A and 1B. Referring toFIG. 2, themodular sensing device150 can receive a mode selection input (200). In some examples, the mode selection input can comprise the user physically pushing ananalog mode selector155 on themodular sensing device150. Additionally or alternatively, the mode selection input can comprise a specified gesture performed while using themodular sensing device150.
• In response to the mode selection input, themodular sensing device150 can interpret the final state machine reports188 in accordance with the mode selected (205). As provided herein, themodular sensing device150 can operate in any number of modes, with each mode corresponding to controlling a remote device (e.g., drive mode), generating user tasks and feedback in connection with a remote device (e.g., training mode), generating user tasks and feedback in conjunction with another modular sensing device (e.g., playing a real-world game with another user), and/or generating standalone user tasks and feedback. As provided herein, user tasks can comprise instructions or suggestions to the user via the output devices of the modular sensing device in accordance with the selected mode. Such instructions or suggestions can be generated based on a programmed game, in which the user is to utilize themodular sensing device150 to perform gestures and action, search for a certain signal source, cause the remotely operateddevice140 to perform a set of actions, and the like. Furthermore, feedback can comprise reactive output to the tasked actions to be performed by the user. For example, the feedback can comprise audio, visual, and/or haptic responses to actions indicating affirmative or negative completion of such tasks. In one example, the feedback indicates to the user whether an instructed task or gesture (e.g., indicated by thesensor data167 and correlated state machine) has successfully been performed.
• According to several examples, the final state machine reports188 can be correlated to a specified output. Thus, based on each finalstate machine report188 themodular sensing device150 can generate commands in accordance with the mode and, when relevant, the sub-mode of the device150 (220). As provided herein, such generated commands can include output commands164 to output audio (222), visual (226), and/or haptic (224) output on themodular sensing device150. Additionally or alternatively, themodular sensing device150 can generate control commands162 (228) for operating a remotely operateddevice140. In either case, themodular sensing device150 can transmit or execute the commands accordingly (225).
• At any given time in any given mode, themodular sensing device150 can identify asub-mode trigger186 in the final state machine reports188 (210). In response to thesub-mode trigger186, themodular sensing device150 can reconfigure interpretation of one or more final reports from one or more corresponding state machines (215). Based on each finalstate machine report188, themodular sensing device150 can generate commands in accordance with the mode and the sub-mode of the device150 (220), including the altered interpretations based on the sub-mode trigger(s)186. As discussed above, such generated commands can include output commands164 to output audio (222), visual (226), and/or haptic (224) output on themodular sensing device150. Additionally or alternatively, themodular sensing device150 can generate control commands162 (228) for operating a remotely operateddevice140. In either case, themodular sensing device150 can transmit or execute the commands accordingly (225).
• Modular Sensing Device
• FIG. 3 illustrates an example modular sensing device insertable into a plurality of compatible apparatuses. Themodular sensing device300 shown inFIG. 3 can comprise various components and modules of themodular sensing devices100,150 shown in the block diagrams ofFIGS. 1A and 1B. Referring toFIG. 3, themodular sensing device300 can include a number of output devices, such as anLED array310, an audio output device320 (e.g., a speaker), and a haptic driver360 (included within the device). Furthermore, themodular sensing device300 can include amode selector330, which can comprise an analog or digital button to enable the user to select a particular mode of thedevice300 by, for example, scrolling through a stored series of modes. Themodular sensing device300 can further include memory andprocessing resources365 that can execute the selected mode (either in the state machine implementation (FIG. 1B) or the executed instruction set implementation (FIG. 1A) described herein).
• In various aspects, themodular sensing device300 also includes a communications interface370 (e.g., a BLUETOOTH low energy, WiFi, WiGig, WiMAX, or cellular radio interface), and anIMU340 to provide the memory andprocessing resources365 with sensor data for detecting gestures performed by the user. As described herein, depending on the mode and sub-mode of thedevice300 the memory andprocessing resources365 interpret the sensor data to generate outputs via theoutput devices310,320,360 and/or commands or responses to be output to a connected device via the communications interface370 (e.g., a remotely operated device or another modular sensing device). Furthermore, in some implementations, themodular sensing device300 can include an input interface350 (e.g., a mini-USB port) to enable charging of one or more batteries and/or uploading of additional mode instructions. In variations, themodular sensing device300 can include an induction interface to charge one or more batteries and/or to enable inductive pairing with a second device to establish a wireless communications link.
• In the various examples described herein, themodular sensing device300 can be insertable into or otherwise attachable to any number of compatible apparatuses, such as wearable devices395 (wrist devices, rings, pendants, hats, glasses, etc.) wielded devices385, companion toys or dolls, and the like. Furthermore, themodular sensing device300 can be implemented in various other form factors, can be sewn into clothing, or can be mounted, glued, or otherwise attached to various apparatuses. Such apparatuses can each include amodule holder387,397 into which themodular sensing device300 may be inserted or otherwise mounted or attached. Thus, according to examples provided herein, the user can utilize the apparatuses into which themodular sensing device300 has been inserted or attached, to perform various gestures in accordance with a selected mode of themodular sensing device300.
• Finder and Mining Mode
• FIG. 4 is a block diagram illustrating an example portable sensing device operable as a signal detection system for acquiring virtual or digital resources. In more detail, awearable device400 may be a standalone device or may be incorporated with a mobile computing device running an application specific to detecting and recording signals from network devices, and searching for those network devices associated virtual resources. Standalone devices may include a pocket device, such as a key fob, a wrist-worn device or other wearable device (as described herein), and the like. For mobile computing environments, thewearable device400 may utilize mobile computing device resources, such as device hardware/firmware, or operate utilizing a combination of hardware and software. For example, thewearable device400 may be enabled upon initiation of an application on the mobile computing device. Additionally or alternatively, thewearable device400 may continue to run during hibernation or sleep mode of the application using background or standby resources. Alternatively, thewearable device400 may be enabled in thefinder mode406 andmining mode407 upon user input on amode selector405 of thewearable device400, as shown and described with respect toFIG. 1. Selection and execution of the finder and/or mining mode can generate anexecution command408, which can enable the processing resources of thewearable device400 to execute the respective mode instructions for translating sensor data and providing feedback responses.
• Referring toFIG. 4, thewearable device400 can include asignal interface410 to detect emitted signals from any number of network-enableddevices460. Such emitted signals may be background signals advertising the presence and media access control (MAC) addresses ofsuch devices460. For example, at time t1, thewearable device400 may be within wireless range of alaptop computing device462 having aunique identifier UID1. The UID of the laptop computing device462 (i.e., UID1) may correspond to the MAC address of thelaptop computing device462. Thewearable device400 can include aUID recorder420, which can receiveUIDs412 from thesignal interface410 and record therespective UIDs412 in aUID log432 of alocal memory resource430. Accordingly, theUID recorder420 can log the laptop computing device's462 UID (i.e., UID1) in theUID log432.
• As a further example, at time t2, thewearable device400 may come within wireless range of anaccess point463 having aMAC address UID2. According to examples described herein, thesignal interface410 can receive the MAC address of theaccess point463. For example, thesignal interface410 can receive a beacon from theaccess point463 which includes the access point's463 MAC address (UID2). Thesignal interface410 can communicateUID2 to theUID recorder420, which can logUID2 in theUID log432.
• According to some examples, thewearable device400 can establish a network connection, e.g., vianetwork480, with ahost server470. Upon connecting with thehost server470, thewearable device400 can transmitUID log data482 from the local UID log432 to thehost server470. Accordingly, thehost server470 can compile theUID log data482 and associate therespective UIDs412 from thelog data482 with a user account476 associated with a user of thewearable device400. In such examples, theremote host server470 can perform any number of functions described herein. For example, thehost server470 can utilize theUID log data482 from thewearable device400 to identify the network devices (i.e., thelaptop computing device462 and the access point463) with which thewearable device400 came within wireless range. Thehost server470 can store auniversal association list472 that may list associations between, for example,UID1 of thelaptop computing device462 and a specified virtual resource. Accordingly, thehost server470 can attribute an amount of the specified virtual resource associated withUID1 to the user account476 associated with thewearable device400. Thus, if the specified virtual resource associated withUID1 was, for example, a virtual ore, then thehost server470 could attribute a predetermined amount of virtual ore to the user account476.
• Additionally or alternatively, thehost server470 may perform a lookup in the universal associations list472 and identify that UID2 (corresponding to the access point463) does not yet have an associated virtual resource. Thus, thehost server470 may select, either sequentially or randomly, a virtual resource from a virtual resource catalog474 and log a new association between the selected virtual resource andUID2 in theuniversal associations list472. Accordingly, an amount of the new selected resource can be attributed to the user account476, in which respective virtual resources may be accumulated based on wireless detection events between thewearable device400 and thenetwork devices460.
• Additionally or alternatively, one or more operations described with respect to thehost server470 may be performed by thewearable device400 itself. For example, thewearable device400 can include anassociation engine440 which, upon detection of a wireless signal from anetwork device460, can determine whether that device is locally associated with a virtual resource. Thus, thememory resource430 of thewearable device400 can include an association table434 which may includepredetermined associations435 between thenetwork devices460 and virtual resources. As an example, at time t3, thewearable device400 can come within wireless range of amobile device464, which can announce its presence usingMAC address UID3. Thesignal interface410 can receiveUID3, and theassociation engine440 can perform a lookup in the association table434 to determine whetherUID3 is associated with a particular virtual resource (e.g., virtual oil). Thus, thewearable device400 may flag the association betweenUID3 and virtual oil until a connection with thehost server470 is established to attribute the virtual oil to the user account476.
• Alternatively, thewearable device400 can include aresource allocator450 and thememory resource430 can include avirtual resource log436, comprising an accumulation of the virtual resources collected by the user. Theassociation engine440 can transmit theassociations435 to theresource allocator450, which in turn, can select and compile respective amounts of accumulated virtual resources (e.g., five units of virtual oil corresponding to theUID3 detection event) in thevirtual resource log436.
• According to one or more examples, at time t4, thewearable device400 may come within range of atablet computing device465.UID4, associated with the MAC address of thetablet computing device465 may be received by thesignal interface410 and communicated to theassociation engine440, which can determine from the association table434 thatUID4 is associated with, for example, virtual coal. Theassociation engine440 can communicate theUID4 association to theresource allocator450, which can log an amount of virtual coal in thevirtual resource log436. Additionally or alternatively, if the virtual resources (e.g., virtual coal) of all recently detecteddevices460 are logged in thevirtual resource log436, the signal detection system can flush the UID log432, since the UIDs of thenetwork devices460 are already known and associated in the association table434.
• Upon network connectivity with thehost server470 over thenetwork480, the signal detection system can transmit virtualresource log data484 to thehost server470 to enable thehost server470 to attribute collected virtual resources to the user account476 associated with the user of thewearable device400. Various alternatives are contemplated. For standalone devices, thewearable device400 can communicatively couple with a user's computing device, such as a smartphone or tablet computer. The virtualresource log data484 may then be communicated to the user's computing device which can, in turn, attribute the collected virtual resources to the user's account locally or via network connection to thehost server470. The collected virtual resources may be expended as a result of gameplay, or via various incentive-based programs increasingly prevalent in the mobile application marketplace to induce users to interact with stipulated content.
• Additionally or alternatively, the consumption, accumulation, and/or expenditure of collected virtual resources may be performed dynamically in connection with task-oriented operations (e.g., gameplay in the game mode) on amobile computing device495 running a respective application utilizing such virtual resources. Further, a standalonewearable device400 may incorporate near field communication (NFC) technology such that the supplementary task of collecting virtual resources may be performed during a user's routine daily activities, and the user can compile the virtual resources via NFC link with the user'scomputing device495.
• In accordance with the above examples, at time t5, thewearable device400 may come within range of amobile device466 having a unique identifier (e.g., MAC address)UID5. Thesignal interface410 can communicate the mobile device's466 identifier to theUID recorder420 which can recordUID5 in the UID log432 of thelocal memory resource430. The mobile device's466 identifier may be further communicated to theassociation engine440 which can perform a lookup in the association table434 to determine whetherUID5 is associated with a virtual resource.
• IfUID5 is not associated with a virtual resource, thewearable device400 can subsequently utilize the UID log432 to communicateUID log data482 to thehost server470 to receive an association betweenUID5 and a virtual resource. Alternatively, thewearable device400 can include a local virtual resource catalog, similar to the virtual resource catalog474 of thehost computer470, and can perform a sequential or randomized selection of a virtual resource (e.g., virtual wood) to associate withUID5.
• IfUID5 has already been associated with a virtual resource (e.g., virtual wood), as resultant from the lookup performed by theassociation engine440, then theassociation435 can be communicated to theresource allocator450, which can allocate a predetermined amount of virtual wood in thevirtual resource log436. Once a connection between thewearable device400 and thehost server470 is established, the virtualresource log data484, which can comprise the accumulated virtual resources since the last established connection, can be communicated to thehost server470 for attribution to the user's account476. Once the virtualresource log data484 is communicated, thevirtual resource log436 in thewearable device400 can be flushed.
• Various limitations to resource allocation are realized. For example, thesignal interface410 of thewearable device400 may be restricted to only receive virtual resources associated withnetwork devices460 upon establishing a connection with the respective device, as opposed to merely detecting thenetwork device460. Furthermore, the allocated amounts of a particular virtual resource associated withmultiple network devices460 may be diverse. For example, theresource allocator450 may allocate more virtual resources when thewearable device400 comes within contact of a less accessible access point (e.g., an access point at a remote location). Furthermore, at any given expiration time (e.g., after an expiration period or a time-limited gaming session), the UID log432, the association table434, and/or thevirtual resource log436 may be reset or otherwise reconfigured.
• Examples described with respect toFIG. 4 illustrate resource association and allocation being performed by thewearable device400. However, according to some aspects, themobile computing device495 can perform various processes as described with regard to thewearable device400. For example, themobile computing device495 can receive theUID data484 for each detectednetwork device460, and can locally perform lookups for the associations and allocations of the virtual resources. Accordingly, thememory430 shown as a component of thewearable device400, may be included on themobile computing device495 and/or accessible by themobile computing device495, running a designated application, in thehost server470.
• As further described herein, upon detecting aparticular network device460 associated with a particular virtual resource in the finder mode, thewearable device400 can initiate the feedback mechanism to notify the user that virtual resources are proximate to the user's current location. The feedback output can include haptic feedback in combination with lighting and or audio feedback. In many aspects, the user can be notified and enabled to select the mining mode, in which the user can “mine” the virtual resource from thenetwork device460. In the mining mode, the feedback mechanism can be initiated and adjusted dynamically to enable the user to search for a direction towards the virtual resource to acquire a certain allocation, and/or to locate the exact position of the network device to acquire a certain allocation, as described below with respect toFIG. 6.
• In still further implementations, thewearable device400 can utilize location-based resources (not shown) in order to identify virtual resources associated with a particular waypoint. For example, a gameplay environment may preconfigure virtual resources in GPS locations in the real world. Thewearable device400 can identify such locations (e.g., when thewearable device400 is within a certain proximity of a particular waypoint) and provide feedback via a feedback mechanism that enables the user to search for and mine the virtual resource associated with the waypoint.
• FIG. 5 is a block diagram illustrating an example virtual resource management system utilized by a portable sensing device (e.g., wearable device400), or amobile computing device495 executing an application in connection with awearable device400. The virtualresource management system500 can include features from thewearable device400 described with respect toFIG. 1A. Furthermore, one or more components of the virtualresource management system500 may be comprised in any number of electronic devices, such as smartphones, tablet computers, personal computers, laptop devices, wearable computing systems, and the like. Accordingly, the virtualresource management system500 can comprise an application-based program running on any of such devices, enabling a user to collect, accumulate, expend, and consume virtual resources in connection with task-oriented operations performed on the device. For example, the user's device may run an application specific to virtual resource gameplay, rendering agaming environment578 on adisplay590. The user may performinteractions594 using thedisplay590 in order to utilize compiled virtual resources in thevirtual resource log565, as shown inFIG. 5.
• Additionally or alternatively, the virtualresource management system500 shown and described with respect toFIG. 5 can be provided withUIDs512 from, for example, a wearable device in a finder mode that operates to locate virtual resources as a user walks or otherwise travels across various networks and comes within network range of various network devices. In some aspects, the wearable device can transmit UIDs of various detected network devices to the mobile computing device, which can connect with thehost server570 to determine whether the detected network devices are associated with virtual resources. If so, the mobile computing device can transmit a confirmation signal to the wearable device which can provide feedback to the user (e.g., audio, haptic, and/or visual feedback) indicating that virtual resources are nearby. The mobile computing device or the wearable device itself can determine a direction towards the network device associated with the virtual resource. Furthermore, the generated feedback on the wearable device can enable the user to select the mining mode, allowing the user to first identify the direction towards the virtual resource(s) (e.g., via dynamically provided feedback indicating whether the user is “hot” or “cold” depending on the direction the user points the wearable device), and optionally locate the actual position of the network resource device in order to be allocated with a certain amount of the virtual resource. In some aspects, the wearable device can include an IMU to determine the user's orientation and a direction in which the user is facing in order to provide the feedback. The mining mode may terminate automatically upon acquiring the virtual resources, and the wearable device may revert, automatically, back to finder mode.
• Referring toFIG. 5, the virtualresource management system500 can include a signal detector510 (e.g., the wearable device) to detectUIDs512 of various network devices, as discussed above with respect toFIG. 4. TheUIDs512 may be logged in aUID Log530 by aUID module520. As shown inFIG. 5, various UIDs (i.e.,UID XYZ532,UID EFG534,UID NOP536,UID TUV538, and UID DK539), each associated with a respective network device, have been logged in theUID log530. TheUIDs512 may be communicated to anassociation engine540, which can perform lookups in an association table560 to determine whether a respective UID is associated with a respective virtual resource. Thus, in the example shown inFIG. 5,UID XYZ532 is associated with the virtual resource,virtual wood562,UID EFG534 is associated withvirtual ore564,UID NOP536 is associated withvirtual food items566, andUID TUV538 is associated withvirtual workers568.
• Virtual resources can further include various forms of digital currency or money, or images, video clips, three-dimensional (3D) interactive models, sound clips, a mini-game, or other content that may be presented to the user in response to finding the virtual resource. Thus, the association table560 can be provided with any type of virtual resource described herein. Furthermore, virtual resources may be stored for subsequent task-oriented activities (e.g., virtual gold or virtual money to be expended in a subsequent game), or may require immediate consumption (e.g., a “mined” video clip discovered by the user which can be viewed once and deleted).
• In accordance with examples described herein, when thesignal detector510 detects say, an access point corresponding toUID NOP536,UID NOP536 may be communicated to theassociation engine540, which, upon performing a lookup in the association table560, can determine thatUID NOP536 is associated withvirtual food items566. Theassociation engine540 can communicate thisassociation542 to aresource engine550, which can logallocations552 of virtual resources in avirtual resource log565, as shown inFIG. 5. Thus, once theassociation engine540 communicates the association ofUID NOP536 withvirtual food items566 to theresource engine550, theresource engine550 can allocate a predetermined amount ofvirtual food566 in thevirtual resource log565. As shown in thevirtual resource log565 ofFIG. 5, the user has accumulated1777 units ofvirtual food566, in addition to765 units ofvirtual wood562,925 units ofvirtual ore564, and25 units ofvirtual workers568. However, the user has yet to accumulate any virtual crystal, virtual coal, or virtual oil, which may be essential items in connection with the renderedgameplay592 on thedisplay590.
• Continuing with examples provided herein, as the user comes within range of say, a remote access point associated withUID IJK539, thesignal detector510 can communicateUID IJK539 to theassociation engine540, which upon performing a lookup in the association table560, can determine thatUID IJK539 is not yet associated with a virtual resource (i.e., unknown569). In accordance with examples described herein, theassociation engine540 may attributeUID IJK539 with a particular virtual resource (e.g., virtual oil) by referencing a local virtual resource catalog, similar to the virtual resource catalog574 of thehost server570. Alternatively, theassociation engine540 may compile anassociation call544, which can be communicated to thehost server570 upon establishing a network connection vianetwork580. Thehost server570 can sequentially or randomly select a virtual resource (e.g., virtual oil), from the virtual resource catalog574, and log the new association betweenUID IJK539 and virtual oil in auniversal associations list572. Thehost server570 can respond to association calls544 withnew associations596 logged in theuniversal associations list572. Thenew associations596 can be communicated to the virtualresource management system500 over thenetwork580 via acommunication interface595 of the virtualresource management system500. Accordingly, thenew associations596 can be received by theassociation engine540, which can log thenew associations596 in the association table560.
• Thus, in the example described, the association betweenUID IJK539 and virtual oil may be transmitted through thecommunication interface595 of the virtualresource management system500 to theassociation engine540, which can update the association table560 to replace “unknown569” with virtual oil.
• Furthermore, theresource engine550 can manage theresource log565 based on bothuser interactions594 performed via the renderedgameplay592 on thedisplay590, and detectedUIDs512 and receivedassociations542 based onsuch UIDs512. In various implementations, theuser interactions594 in connection with the renderedgameplay592 can cause virtual resources stored in thevirtual resource log565 to be expended.Such expenditures567 in the renderedgameplay592 can correlate to a depletion of the relevant virtual resource in thevirtual resource log565. Various related examples are contemplated. For example, theuser interactions594 can cause respective virtual resources to be traded, consumed, accumulated, invested, or expended in accordance with the renderedgameplay592.
• In the provided example above, the detection ofUID IJK539—now associated with virtual oil—can cause theresource engine550 to attribute a predetermined amount of virtual oil (e.g., five units) in thevirtual resource log565. Furthermore, as shown inFIG. 5,user interactions594 in connection with the renderedgameplay592 can cause dynamic updates, by theresource engine550, to thevirtual resource log565. Thevirtual resource log565 can be dynamic in nature, in connection with the renderedgameplay592 and the detection ofUIDs512. Theresource engine550 can continuously performallocations552 andexpenditures567 of virtual resources in thevirtual resource log565. Additionally or alternatively, theresource engine550 can communicate accumulatedvirtual resources597 to thehost server570, which may attribute the accumulatedvirtual resources597 to the user's account576 to save progress data corresponding to the renderedgameplay592.
• The renderedgameplay592 may be application or software-based utilizing any number of resources of the user's computing device. The renderedgameplay592 may be provided in connection with augmented reality, virtual reality, or a virtually generatedgaming environment578 provided by thehost server570. For augmented reality implementations, thegaming environment578 provided can comprise virtual features rendered in a real world environment. For example, a camera included on the user's computing device may be utilized to capture real-world images or video, and thegaming environment578 may be rendered thereon. Furthermore, the renderedgameplay592 may be incorporated in conjunction with the use of a remotely operated self-propelled device. In such implementations, the rendered gameplay may include virtual controls to remotely control the self-propelled device. Accordingly, the virtualresource management system500 can operate in connection with the self-propelled device, which can incorporate one or more features of the virtualresource management system500, such as the signal detection features (i.e.,signal detector510,UID module520, and UID log530).
• Finder and Mining Mode Methodology
• FIG. 6 is a flow chart describing an example method of acquiring virtual resources using a wearable device. In the below discussion ofFIG. 6, reference may be made to like reference characters representing various features ofFIG. 4 for illustrative purposes. Furthermore, the method described in connection withFIG. 6 may be performed by, for example, thewearable device400 as illustrated inFIG. 4, or awearable device400 in communication with amobile computing device495 running a designated application, as shown inFIG. 4. Referring toFIG. 6, thewearable device400 can initiate a finder mode and initially detect a signal, such as an advertising beacon, from a network device, such as an access point (600). The beacon can include the network device's MAC address, or other UID representing the network device.
• Thewearable device400 can then perform a lookup in an association table434 to determine whether the UID is associated with a given virtual resource (605). In some aspects, thewearable device400 can transmit the UID to themobile computing device495, which can make the determination accordingly. At decision block (610), the wearable device400 (or mobile computing device495) identifies from the association table434 whether the UID is associated. If the UID is not associated with a virtual resource (614), then thewearable device400 can continue to monitor and detect signals from network devices (600), and/or generate a request for a UID association and retrieve the virtual resource association. However, if the UID is associated with a given virtual resource (612), then thewearable device400, or user utilizing thewearable device400, can be allocated a predetermined amount of the given virtual resource in thevirtual resource log436 in the following manner.
• Upon determining that the network device is associated with a virtual resource, thewearable device400 can generate feedback to the user indicating the proximate virtual resource (615). Thewearable device400 may then receive a user input (e.g., on an input mechanism or a mode selector on the wearable device) to search for the virtual resource, thereby initiating the mining mode (420). In the mining mode, thewearable device400 can determine a direction towards the network signal (e.g., the advertising beacon having the UID associated with the virtual resource) (625). Then thewearable device400 can monitor the sensor data, from one or more sensors of thewearable device400, to determine whether the user points the wearable device towards or away from the virtual resource signal (630). While the user points thewearable device400 in differing directions, thewearable device400 can adjust the feedback dynamically (635).
• For example, the feedback, which can comprise a combination of haptic, audio, and visual feedback, can increase in intensity as the user points thewearable device400 towards the network device, and decrease in intensity as the user points thewearable device400 away from the network device. Accordingly, once the user has pointed thewearable device400 in the correct direction for a predetermined time period (e.g., three seconds), thewearable device400 can acquire a certain allocation of the virtual resource, and store or otherwise enable the user to consume or expend the acquired resource (640).
• FIGS. 7A and 7B are low level flow charts describing example processes for managing virtual resources in connection with signal detection and gameplay. In the below discussion ofFIGS. 7A and 7B, reference may be made to like reference characters representing various features ofFIG. 5 for illustrative purposes. Furthermore, the low level method described in connection withFIGS. 7A and 7B may be performed by, for example, the virtualresource management system500 as illustrated inFIG. 5. Referring toFIG. 5, the virtualresource management system500 can detect advertising signals (e.g., beacons) from network devices (700). Such advertising signals may include UIDs, such as MAC addresses for the detected network devices. The virtualresource management system500 can then record the UIDs of the network devices in the UID log530 (705).
• The virtualresource management system500 may perform a lookup in the association table560 (710) to determine whether a respective UID is associated with a respective virtual resource (715). If the respective UID is associated with a respective virtual resource (717), then the virtualresource management system500 can allocate a predetermined amount of the respective virtual resource in the virtual resource log565 (720). Thereafter, the process may begin again with the detection of advertising signals from network devices (700). However, if the respective UID is not associated with a respective virtual resource (719), the virtualresource management system500 can generate a request, or anassociation call544, to create anew association596 for the respective UID (725).
• The virtualresource management system500 can transmit the request to thehost server570 when a network connection is established (730). Alternatively, the virtualresource management system500 can locally select a respective virtual resource from a local virtual resource catalog (740). Such a selection may be made sequentially (743), for example, if the virtual resource catalog is a sequential list of virtual resources. Or, the selection may be made randomly (741) from the virtual resource catalog by way of a random selection technique.
• In various implementations, the virtualresource management system500 can receive, remotely or locally, associated virtual resources for unassociated UIDs (745). Thus, the respective unassociated UID may be associated with a respective virtual resource, and thereafter the virtualresource management system500 can allocate a predetermined amount of the respective virtual resource in the virtual resource log565 (720). After all detected UIDs are associated with their respective virtual resources, the virtualresource management system500 can flush the UID log747, since storing the UIDs may no longer be necessary.
• The process as described in connection withFIG. 7B may be performed before, after, or in conjunction with the process as described with respect toFIG. 7A. Referring toFIG. 7B, the virtualresource management system500, may receive a user input (e.g., via a touch input on a touch-sensitive display) to launch a gaming application associated with trading, consuming, accumulating, earning, investing, or otherwise expending virtual resources. Based on the user input, the virtualresource management system500 can initiate the gameplay application to place the wearable device in game mode (750). In doing so, the gaming environment may be rendered on the display590 (755) (e.g., on the display of the mobile computing device).
• During gameplay, the virtualresource management system500 can receivevarious user interactions594 in connection with the gameplay (760).Such user interactions594 may be performed by way of touch inputs, mouse interactions, keyboard interactions, interactions using a game controller such as a joystick or a specialized controller device, or a combination of the above. Furthermore,such user interactions594 may be performed in connection with remote operation of a remotely operated device. In accordance with examples described herein, the gameplay can incorporate a virtual or augmented reality environment in which the user can utilize collected virtual resources. For example, collected virtual resources may be consumed by a virtual character under operative control of the user. Additionally or alternatively, selected virtual resources can be expended to build a virtual building or town. Collected virtual food items may be utilized to feed a virtual colony of gameplay characters. Virtual oil may be utilized by the user during gameplay to modernize a primitive society. Virtual workers may be employed for production or to build infrastructure. Various alternatives and additions in connection with task-oriented operations and gameplay are contemplated.
• Accordingly, based on theuser interactions594, theresource engine550 of the virtualresource management system500 can dynamically modify or update thevirtual resource log565 to enable the user to expend, trade, accumulate, invest, earn, etc. virtual resources from the virtual resource log565 (765).
• When the user wishes to end a gameplay session, the virtualresource management system500 may receive a user input to deactivate the gaming application, and thus terminate the gaming mode (770). In order to save gameplay progress, theresource engine550 of the virtualresource management system500 can compile the virtual resources left in thevirtual resource log565, and transmit a saved list of virtual resources to the host server570 (775). Thereafter, the virtualresource management system500 may flush the virtual resource log (777), since the progress is remotely saved.
• Use Case Scenarios
• FIGS. 8A through 8C illustrate unique identifier logs and association tables utilized in connection with virtual resource association, acquisition, and allocation. Various implementations of the disclosed systems and method described herein are contemplated. For example, the detected MAC address of various network devices may be associated with any number of plausible items. Such items may be individuals associated with the network device (e.g., an owner of the network device), or such items may be associated with persons of interest (e.g., characters in connection with a gaming environment), as illustrated inFIG. 8A. Alternatively, such items may be locations of the respective network devices, such that a user is enabled to identify where he/she has traveled, as illustrated inFIG. 8B. Additionally or alternatively, such items may be associated with a time stamp, thereby enabling a user to determine that an interaction has taken place with a particular network device and a time in which the interaction took place, as illustrated inFIG. 8C. Furthermore, the examples provided with respect toFIGS. 8A through 8C may be implemented using thewearable device400 as described with respect toFIG. 4, and/or using a modified virtualresource management system500 as discussed with respect toFIG. 5.
• Referring toFIG. 8A, aUID Log810 of a signal detection system (e.g., a wearable device or wrist-worndevice800 operating in finder mode) can compile UIDs (e.g.,UID XYZ812,UID EFG814,UID NOP816,UID TUV818,UID IJK819, etc.) associated with respective network devices. The respective network devices may be any one of an access point, a mobile computing device, a tablet computer, a personal computer, a BLUETOOTH-enabled device, a radio-frequency identification device, a local area network enabled device, etc. Furthermore, a user may carry the wrist-worndevice800, which can come within wireless detection range of any number of the foregoing network devices. In accordance with examples described herein, the wrist-worndevice800 can compile UIDs corresponding to the network devices in aUID Log810.
• In many examples, the wrist-worndevice800 only compiles UIDs associated with network devices in which the user come within wireless range. In such examples, the user may enable a function on a computing device (e.g., a tablet computer, smart phone, PC, etc.), which can pull the UIDs from the wrist-worndevice800 and make associations according to a local association table. In order to transmit the UIDs to the user's computing device, the wrist-worndevice800 can include connectivity functions to establish a connection with the computing device (e.g., Wi-Fi, BLUETOOTH, etc.). Alternatively, the wrist-worndevice800 can include inductive data communication capabilities in order to transmit the UIDs to the computing device over an inductive link.
• Alternatively, the wrist-worndevice800 can include a local association table820 in order to make a given association when the wrist-worndevice800 comes within wireless range of a given network device. Upon each detection event, the association table820 of the wrist-worndevice800 can log an association. For example, as the user carries the wrist-worndevice800 within wireless range of an access point, the wrist-worndevice800 can receive the access point's advertising beacon, which can include the access point's unique identifier,UID XYZ812. The wrist-worndevice800 association table820 can be referenced to identify that UID XYZ is associated with Ricky H.822—who may be an owner of the access point. The user may then journey within range of a wireless device withUID EFG814, which the association table820 may associate with Jose C.824—an owner of the wireless device. In such implementations, after an excursion in which the user encounters any number of network devices, the user may thereafter review the association table/log to identify the individuals (i.e., Ricky H., Jose C., etc.) whose devices the user encountered during the excursion.
• Alternatively, for task-oriented implementations (e.g., gaming), the association table may associate such UIDs with characters or places in a gaming environment. As contemplated in one or more examples, each detection event can correspond to a meeting or visitation in which the user meets with a character or visits a point of interest. A task-oriented application may require the user to meet a certain character or visit a certain place before a next achievement is reached. Accordingly, along a physical excursion by the user, the wrist-worndevice800 may detect advertising beacons for respective devices associated withUID NOP816,UID TUV818, andUID819. The association table820 may be referenced to identify characters and/or places (real or virtual) with which the respective network devices are associated. As an example, the association table/log820 can identify Mark M.826 as being associated withUID NOP816, Dave. S.828 as being associated withUID TUV818, and theColiseum829 as being associated withUID IJK819. Upon transmitting such characters and/or places to the user's computing device, the task-oriented application can input such meetings and visits and record a number of respective achievements.
• FIG. 8B illustrates another usage scenario in which the UIDs of network devices may be associated with a real-world or mock-world environment. As an example, the user may carry thesignal detection device800 to within wireless range of an access point of a coffee shop, where the access point has a uniqueidentifier UID XYZ832. The wrist-worndevice800 can log UID XYZ in theUID Log830 and reference the association table/log840 to determine thatUID XYZ832 is associated with thecoffee shop842. After an excursion of passing through any number of wireless beacons, the user can map the excursion using the physical locations, logged in the association table/log840, of the network devices along the way.
• In some examples, the physical locations of the network devices may be determined via the detected beacon, which may include location information. Alternatively, the wrist-worndevice800, or mobile computing device to which the wrist-worndevice800 is connected, can include location-based functionality (e.g., GPS resources) to log the location in the association table/log840. Accordingly, in response to a detection event, the wrist-worndevice800 can be triggered to pinpoint the physical location of the detection event—which can be logged along with the UID of the network device, as shown inFIG. 8B.
• In other examples, the location associated with a UID may be based on a mock-world environment corresponding to, for example, a gaming environment. For example, a user may interact with a task-oriented application on a computing device, which may require the user to visitEl Dorado844. The user may then physically search for or journey to a specified network device havingUID EFG834, which, in accordance with the association table, is associated withEl Dorado844. In such examples, the user may be enabled to reference the association table840, which may provide a physical location in the real-world of the network device associated withEl Dorado844. Thus, upon wirelessly contacting the network device havingUID EFG834, the association table/log840 can log the detection event, and the user can accomplish a next achievement in the task-oriented application.
• Additionally, in combination with the above examples with respect toFIGS. 8A and 8B, the wrist-worndevice800 may also include a timer or clock to enter a timestamp as triggered by a detection event, as illustrated inFIG. 8C. Thus, a user may further review a time in which the wrist-worndevice800 detected the wireless signal of a respective wireless device. For example, during a given excursion, the user may come within wireless range of a network device havingUID XYZ852, which can be logged as being associated withItem1862. The detection event ofUID XYZ852 can further trigger a clock or timer to log a timestamp associated with the detection event. The time may reflect a local or universal time, or may reflect an elapsed time from say, the beginning of the excursion or a start time of the task-oriented application.
• Default and Sharing Mode
• FIG. 9 illustrates a wearable device pairing triggering virtual resource data sharing in connection with gameplay, as described herein. Thewearable device900 may be carried or worn by a user to detect advertising beacons or wireless signals from various network devices. The wearable device900 (e.g., as wrist-worn device) can include aUID Log901 and automatically log each UID (e.g., MAC address) corresponding to each wireless device detected. According to many examples, upon linking with acomputing device905,wearable device900 can transmit the UID's904 from theUID Log901 to thecomputing device905. The link may be any data connection. For example, thewearable device900 can include functionality corresponding to Wi-Fi, radio-frequency, infrared, BLUETOOTH, near-field communication (NFC), and the like. Thus, the UID's904 may be transmitted to thecomputing device905 over such a communication link.
• In accordance with examples discussed with respect toFIGS. 8A through 8C, thewearable device900 can collect UIDs corresponding to various network devices, which may be associated with any number of items. For example, an association table of either thecomputing device905 or thewearable device900 can include associations between UIDs of network devices and the registered owners of those network devices. Thus, upon detection of such network devices, the registered owner may be logged in anassociation log903. In such examples, a user may scroll through the association log903 to identify individuals with which the user came into wireless contact.
• In variations, the association table of either thewearable device900 or thecomputing device905 can associate various network devices with real-world or mock-world characters, landmarks, or other places. Such associations may be made in connection with a task-orientedapplication902, such as a gaming application providing a gaming environment.
• In some examples, thewearable device900 can itself include an association table/log and/or a virtual resource log to perform and log associations and collect virtual resources. In such examples, loggedassociation items906 and/or collectedvirtual resources908 may also be transmitted to the computing device over the communication link. Furthermore, timestamps909 correlated to the detection events and logged associateditems906 may also be transmitted to thecomputing device905.
• In similar implementations, thecomputing device905 may run a task-orientedapplication902, which can trigger the communication link with thewearable device900. Execution of the task-orientedapplication902 can correspond to running a game providing a gameplay environment which utilizes items associated with the UID's904 of network devices. The task-orientedapplication902 can cause thecomputing device905 to receive the UID's904 from thewearable device900 and reference anassociation log903 to determine whether a given UID is associated with a given association item. If the given UID is not associated, thecomputing device905 can create an association or retrieve an association from a host server. Given an association, thecomputing device905 can log a specified amount of collected virtual resources in a localvirtual resource log907 for use in the task-orientedapplication902.
• In certain aspects, thewearable device900 may operate in default mode in which thewearable device900 monitors for wireless signals indicating that another wearable device, (e.g., wearable device930) is nearby. In the default mode, thewearable devices900,930 can each detect wireless signals from each other (e.g., beacon signals) and can generate feedback (e.g., a haptic and audio response) indicating such. The users can visually locate each other and have the option of pairing theirwearable devices900,930. As discussed herein, the default mode may be executing as a background mode while thewearable device900 operates in another mode, such as finder mode.
• When thedevices900,930 are paired, for example, via aninductive link925, thewearable devices900,930 can each initiate a sharing mode that enables the users to view and/or share acquired virtual resources. Thecomputing devices905,910 of the users can each run a task-oriented application, or gaming application912 that enable the users to share information regarding the virtual resources collected, and trade virtual resources accordingly. In the example shown, the user of thewearable device930 can receivevirtual resource data927 from the wearable device900 (or the mobile computing device905), and the user'smobile computing device910 can acquire “unlocked”virtual resources929 based on a trade with thewearable device900.
• According to one or more examples, the user may then run the gaming application912 on thecomputing device910 and interact with thecomputing device910 via the gaming application912, which can provide agaming environment902 that requires the use, directly or indirectly, of an association table914 in connection withwearable device930 and the gaming application912. For example, thegaming environment920 may require user interactions with real-world or mock-world characters and/or places, which may be accomplished by coming within wireless range of specified network devices associated with such real-world or mock-world characters and/or places in an association table914 of thecomputing device910. Upon detecting such network devices, thecomputing device910 can log associated items (e.g., real-world characters, mock-world characters, real-world locations, mock-world locations, etc.). In accordance with thegaming environment920, upon logging such associations, various tasks may be achieved.
• In similar examples, thegaming environment920 may require the collection and use of virtual resources, as described above. Thus, the user may run the signal detector/gaming application912 and interact with thegaming environment920, which may require the user to collect a number and amount of virtual resources. In the sharing mode, thewearable device930 enables the user to acquire unlockedvirtual resources929 from another user, which can be compiled in avirtual resource log918 associated with the gaming application912. Accordingly, the user can perform a physical excursion to enable thecomputing device910 to wirelessly interact or otherwise detect various network devices to collect such virtual resources for utilization in thegaming environment920. Thegaming environment920 can provide thevirtual resource log918 which can inform the user of which virtual resources and how many of each virtual resource the user has collected.
• Interactive or Battle Mode
• FIGS. 10A and 10B illustrate a wearable device pairing triggering an interactive mode, or battle mode, between a pair of proximate users1002,1012 as shown inFIG. 10B. The interactive mode can be triggered upon selection of the mode on each of a pair ofwearable devices1010,1030. Additionally or alternatively, aninductive link1025 between thedevices1010,1030 can enable the users to select the interactive mode, as shown inFIG. 10A. For example, thedevices1010 may each be connected to a respective mobile computing device, such asmobile computing device1050 andmobile computing device1060 shown inFIG. 10B. Upon performing theinductive link1025, the users1002,1012 can each select the interactive mode to perform a series of actions and alternate between offensive and defensive sub-modes in the interactive mode.
• Referring toFIG. 10B, users1002 and1012 operate theirwearable devices1010,1030 in interactive mode, which transmitsaction data1048 between thedevices1010,1030. Theaction data1048 can correspond touser gestures1041 performed by the users1002,1012 while thedevices1010,1030 are in interactive mode. In many examples, one device may be in an offensive sub-mode in which the user1002 can perform a number ofoffensive actions1042, such as attack actions, using thewearable device1010.Action data1048 corresponding to theoffensive action1042 can be generated by thewearable device1010, and transmitted to thewearable device1030. Thewearable device1010 can generate feedback to the user indicating whether or not theoffensive action1042 was successful.
• At the same time, the proximate user1012 can performuser gestures1041 corresponding to adefensive action1044 based on theoffensive action1042 performed by the user1002. Whether thedefensive action1044 is effective or ineffective against the offensive action1042 (e.g., a blocking action), thewearable device1030 can generate feedback reflecting such. Thewearable devices1010,1012 can alternate between offensive and defensive sub-modes giving each user successive opportunities to perform offensive1042 anddefensive actions1044. Furthermore, sensor patterns can be preconfigured corresponding to any number ofoffensive actions1042 anddefensive actions1044. Still further, only certain sensor patterns for defensive actions1044 (e.g., a specified type of blocking action) may be effective against a givenoffensive action1042. Each sensor pattern for each offensive1042 ordefensive action1044 can be preconfigured in accordance with a set of rules (e.g., gaming rules) that utilize thewearable devices1010,1030 in connection with such physical gameplay. Thus, the users1002,1012 may previously “train” to perfect thedefensive actions1044 in light of theoffensive actions1042 and vice versa in a training mode described herein.
• During an interactive or battle session, themobile computing devices1050,1060 connected to eachwearable device1010,1030 can maintain a tally or score based on any number of metrics in light of the offensive and defensive interactions between the users1002,1012. Successful or partially successful offensive1042 ordefensive actions1044 may be given respective scores, andunsuccessful actions1042,1044 may be given other scores, or negative scores. Apredetermined threshold score1052 may be set by the interactive session, for example, as agreed upon by the users1002,1012, or in accordance with a particular game selected in the interactive mode. Once reached, the interactive session may end and a winner may be declared. A final result can be displayed on the respectivemobile computing devices1050,1060, and the user's1002 may then choose to initiate another interactive session or conclude the experience.
• FIG. 11 is a flow chart describing an example method of implementing an interactive mode between a pair of user utilizing a corresponding pair of wearable devices. In the example described with respect toFIG. 11, reference may be made to various features shown and described inFIG. 10A andFIG. 10B for illustrative purposes. Referring toFIG. 11, thewearable device1010 can detect aninductive pairing1025 with a proximate wearable device1030 (1100). In response to a user selection, thewearable device1010 can initiate an interactive mode (1105). Initially, thewearable device1010 can determine an ordering or sequence between offensive and defensive sub-modes (1110), and then monitor sensor data foruser gestures1041 performed depending on a present sub-mode.
• In the offensive sub-mode (1175), thewearable device1010 can identifyuser gestures1041 indicating an and offensive attack1042 (1120). Thewearable device1010 can do so by active monitoring or via state machine monitoring, as described herein. Sensor patterns corresponding to each of any number of offensive attack actions can be detected, and cause thewearable device1010 to generate and transmitaction data1048 indicating the particularoffensive action1042 performed by the user1002 to the proximate device1030 (1125). Thewearable device1010 may thereafter receive data from the proximatewearable device1030 indicating whether or not the offensive action was successful (1130) and generate a feedback output accordingly (1135). If theoffensive action1042 was successful, then thewearable device1010 can generate positive feedback (e.g., audio, visual, etc.) (1137). If theoffensive action1042 was unsuccessful or a failure, then thewearable device1010 can generate negative feedback (1139). Upon generating the feedback, thewearable device1010 can switch to the defensive sub-mode (1180).
• In the defensive sub-mode (1180), thewearable device1010 can receive data indicating anoffensive attack action1042 from the proximate wearable device1030 (1150). Thewearable device1010 can monitor sensor data for sensor patterns corresponding touser gestures1041 indicating adefensive action1044, or blocking move, performed by the user1002 (1155). Thewearable device1010 can further determine, using timing and sensor data, whether thedefensive action1044 was successful (1160). For example, the user1002 may be given a predetermined amount of time to perform adefensive action1044 in response to anoffensive action1042 once the action data1048 (indicating the offensive action1042) is received. Accordingly, reception of theaction data1048 can trigger a timer on thewearable device1010 giving the user limited time to perform an appropriatedefensive action1044.
• After transmittingaction data1048 corresponding to thedefensive action1044, thewearable device1010 can generate a feedback output (1165). If thedefensive action1044 is successful and performed within the time threshold, thewearable device1010 can generate positive feedback (1167). However, if thedefensive action1044 was unsuccessful, thewearable device1010 can generate negative feedback (1169), and then transmit successful/unsuccessful data to the proximate device1030 (1170), and tally the score for the interactive session.
• During the interactive session, thewearable device1010 can determine whether the threshold score has been achieved by either the user1002 or the proximate user1012 (1185). If not (1187), then the users1002,1012 can continue in the interactive mode, alternating between offensive (1175) and defensive (1180) sub-modes. However, if the threshold score has been achieved by one of the users1002,1012 (1189), then thewearable device1010 can exit the interactive mode and transmit the final scores to the connected mobile computing device1050 (1195).
• Use of multiplewearable devices1010,1030 may be further expanded into various proximity-based activities. Accordingly, in some aspects of interactive mode, multiple users can join and engage in an interactive session that utilizes proximity sensing resources on the wearable devices. Proximity sensors can include components of an inertial measurement unit (IMU) incorporated into eachwearable device1010,1030 (e.g., an infrared sensor). Additionally or alternatively, GPS resources on thewearable devices1010,1030 themselves or on connectedmobile computing devices1050,1060 can be utilized for proximity-based activities. Such activities can include sporting activities to determine, for example, finish line positions of a running race or physical game activities for children (e.g., hide-and-seek or capture the flag). Feedback can be provided to each user in the interactive session, providing information connected to the proximity-based activity (e.g., information indicating a team-leader, a ranking, a team score, positional information, and the like).
• Training Mode
• FIG. 12 is a flow chart describing an example method of initiating a training mode on a wearable device in connection with a self-propelled device. In the below discussion ofFIG. 12, reference may be made to like reference characters representing various features described with respect toFIG. 1A for illustrative purposes. Referring toFIG. 12, thewearable device102 can detect a user input placing thewearable device102 in training mode (1200). The user input can be detected via a mode selector on the wearable device102 (1202), or via launch of a designated application on a connected mobile computing device (1204). Alternatively, the training mode can be initiated via a combination of user inputs on the mode selector and an inductive link with a remotely operateddevice140. Accordingly, thewearable device102 may also detect an inductive pairing with a remotely operated device140 (1205). In response to the inductive pairing, thewearable device102 can transmit data to initiate the training mode on the remotely operateddevice140 as well (1210). The transmitted data can cause the remotely operateddevice140 to execute instructions to aid the user in training for a series of actions. For example, the series of actions can correspond to offensive and defensive actions that the user can implement when thewearable device102 is in interactive or battle mode. Additionally or alternatively, the series of actions can get progressively more difficult as the user successively accomplishes each action.
• Initially, thewearable device102 can synchronize directionally with the remotely operated device140 (1215). In some aspects, the user can manually synchronize the gyroscopic sensors of thedevices102,140 by physically pointing a forward operational direction of the remotely operateddevice140 away from thewearable device102 and providing a calibration input (e.g., an input on the mode selector of the wearable device102) (1217). In other aspects, the gyro synchronization may be performed automatically (1219).
• Automatic synchronization can be initiated by thewearable device102 by generating and transmitting a spin command to the remotely operateddevice140, which can execute a spin accordingly (1220). Using a signal detector, thewearable device102 detect an asymmetry in the radiation pattern of the remotely operateddevice140 as it spins, indicating the direction towards the remotely operated device140 (1225). With a known direction, thewearable device102 can transmit a direction calibration command to the remotely operateddevice140 indicating the direction, which the remotely operateddevice140 can process to align its internal drive system accordingly (1230).
• In many aspects, thewearable device102 can track the location of the remotely operateddevice140 as it traverses and maneuvers (1240). The remotely operateddevice140 can include surface features or an accessory (e.g., a magnetically coupled attachment) that indicates a forward “looking” direction of the remotely operateddevice140. In certain examples, the user is instructed to walk or run around in a circle until the user is directly facing the forward looking direction of the remotely operateddevice140. In many aspects, thewearable device102 can include sensors to determine an orientation of the user. For example, thewearable device102 can determine whether the user is facing an instructed direction in connection with the training mode, such as facing the remotely operateddevice140. Additionally or alternatively, thewearable device102 can generate an output, via the feedback mechanism, instructing the user to perform a set of actions (1245). The output may be in the form of audio instructions, and can be based on data received from the remotely operated device140 (1247), or from thewearable device102 utilizing a local routine set (1249), which may be randomized or sequenced in accordance with the executing training mode instructions.
• Once the instructions are outputted to the user, thewearable device102 can initiate a timer (1250). The timer can be initiated for each instruction outputted to the user, and a threshold time limit can be set for each instruction. Thus, the user can be instructed to perform the set of actions within the predetermined time period. Thewearable device102 can monitor the sensor data to determine whether the user successfully performs the set of actions (1255). Specifically, for each instruction output, thewearable device102 can determine whether the user has performed the instructed set of actions within the established threshold time limit (1260). If so (1261), then thewearable device102 can generate another output instructing the user to perform another set of actions (1245). However, if the user fails to perform the set of actions (1263), then thewearable device102 can terminate the training session and generate a final score (1265), which may be displayed on the user's mobile computing device.
• Sword or Wield Mode
• FIG. 13 is a flow chart describing an example method of implementing a wearable device in a sword mode. In the below description ofFIG. 13, reference may be made to like reference characters representing various features ofFIG. 1A andFIG. 1B. Referring toFIG. 13, thewearable device102 can detect a user input placing thewearable device102 in a sword mode (1300). When in the sword mode, thewearable device102 can initially monitorsensor data137 to determine whether the user has grabbed or grasped an object (1305). In some implementations, a state machine in thememory180 to thewearable device102 specific to the grabbing/grasping action can provide a state machine report to theoutput generator160 indicating that the user is holding an object.
• In many examples, thewearable device102 can determine a series of actions performed by the user with the object from the sensor data137 (1310). Such actions can be actively determined by theoutput generator160 in the sword mode, or via state machine reports from respective state machines that indicate each specific action. In any case, in response to each action, thewearable device102 can generate a feedback response oroutput132 using a feedback mechanism (1315). Thefeedback output132 for each action can be distinct audio responses196 (1317), haptic responses192 (1316), and/or visual responses194 (1318).
• In many examples, based on the action performed by the user, the feedback mechanism can generate a corresponding output, such as sword fighting sounds, or other weapon-like or wand-like sounds based on the executing sword mode instructions. Thus, thefeedback output132 based on the sensor patterns detected when the user is grasping an object may be any such sounds corresponding to the actions performed by the user. In this sense, the “sword mode” may rather be a “wand mode,” a “magic mode,” a “conductor mode,” a “sporting mode” (e.g., for a tennis-like game), and the like. When the user wishes to end the sword mode, thewearable device102 can detect a user input, such as an input on the mode selector110 (1320). In response to the input, thewearable device102 can deactivate or terminate the sword mode (1325).
• Hardware Diagrams
• FIG. 14 is a block diagram that illustrates a computer system upon which examples described may be implemented. For example, one or more components discussed with respect to the remotely operated device ofFIG. 1A, and the methods described herein, may be performed by thesystem1400 ofFIG. 14.
• In one implementation, thecomputer system1400 includesprocessing resources1410, amain memory1420,ROM1430, astorage device1440, and acommunication interface1450. Thecomputer system1400 includes at least oneprocessor1410 for processing information and amain memory1420, such as a random access memory (RAM) or other dynamic storage device, for storing information and instructions to be executed by theprocessor1410. Themain memory1420 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by theprocessor1410. Thecomputer system1400 may also include a read only memory (ROM)1430 or other static storage device for storing static information and instructions for theprocessor1410. Astorage device1440, such as a magnetic disk or optical disk, is provided for storing information and instructions. For example, thestorage device1440 can correspond to a computer-readable medium that store instructions performing sensor data processing and translation operations as discussed herein.
• Thecommunication interface1450 can enablecomputer system1400 to communicate with a computing device and/or wearable device (e.g., via a cellular or Wi-Fi network) through use of a network link (wireless or wired). Using the network link, thecomputer system1400 can communicate with a plurality of devices, such as the wearable device, a mobile computing device, and/or other self-propelled devices. Themain memory1420 of thecomputer system1400 can further store thedrive instructions1424, which can be initiated by theprocessor1410. Furthermore, thecomputer system1400 can receivecontrol commands1462 from the wearable device and/or mobile computing device. Theprocessor1410 can execute thedrive instructions1424 to process and/or translate the control commands1462—corresponding to user gestures performed by the user—and implement the control commands1452 on the drive system of the self-propelled device.
• Additionally, themain memory1420 can further includemode instructions1424, which theprocessor1410 can execute to place the self-propelled device in one or multiple modes to interact with the wearable device. In some examples, execution of themode instructions1422 can place the self-propelled device in an operational mode that providesfeedback1452 and/orinstructions1454 to the wearable device over the network1480 (e.g., in training mode).
• Examples described herein are related to the use ofcomputer system1400 for implementing the techniques described herein. According to one example, those techniques are performed bycomputer system1400 in response toprocessor1410 executing one or more sequences of one or more instructions contained inmain memory1420. Such instructions may be read intomain memory1420 from another machine-readable medium, such asstorage device1440. Execution of the sequences of instructions contained inmain memory1420 causesprocessor1410 to perform the process steps described herein. In alternative implementations, hard-wired circuitry and/or hardware may be used in place of or in combination with software instructions to implement examples described herein. Thus, the examples described are not limited to any specific combination of hardware circuitry and software.
• FIG. 15 is a block diagram that illustrates a mobile computing device upon which examples described herein may be implemented, such as themobile computing device495 ofFIG. 4. In one example, thecomputing device1500 may correspond to, for example, a cellular communication device (e.g., feature phone, smartphone, etc.) that is capable of telephony, messaging, and/or data services. In variations, thecomputing device1500 can correspond to, for example, a tablet or wearable computing device.
• In an example ofFIG. 15, thecomputing device1500 includes aprocessor1510,memory resources1520, a display device1530 (e.g., such as a touch-sensitive display device), one or more communication sub-systems1540 (including wireless communication sub-systems), input mechanisms1550 (e.g., an input mechanism can include or be part of the touch-sensitive display device), and one or more location detection mechanisms (e.g., GPS component)1560. In one example, at least one of thecommunication sub-systems1540 sends and receives cellular data over data channels and voice channels.
• Thememory resources1520 can store a designated control application1522, as one of multiple applications, to initiate thecommunication sub-system1540 to establish one or more wireless communication links with the self-propelled device and/or a wearable device. Execution of the control application1522 by theprocessor1510 may cause a specified graphical user interface (GUI)1535 to be generated on thedisplay1530. Interaction with theGUI1535 can enable the user to calibrate the forward directional alignment between the self-propelled device and thecomputing device1500. Furthermore, theGUI1535 can allow the user to initiate a task-oriented operation (e.g., a game) to be performed by the user in conjunction with operating the self-propelled device with user gestures using the wearable device, as described herein.
• FIG. 16 is a block diagram of an example portable sensing device upon which examples described herein may be implemented, such as thewearable device102 ofFIG. 1A.
• In an example ofFIG. 16, theportable sensing device1600 includes aprocessor1610,memory resources1620, a feedback mechanism1630 (e.g., audio1632, haptic1633, visual1631 devices), a communication sub-systems1640 (e.g., wireless communication sub-systems such as BLUETOOTH low energy), one or more sensors1660 (e.g., a gyroscopic sensor or accelerometer) and an input mechanism1650 (e.g., an analog or digital mode selector). In one example, thecommunication sub-system1640 sends and receives data over one or more channels.
• Thememory resources1620 can storemode instructions1623 corresponding to a plurality ofcontrol modes1622, as described herein, which can be executed by theprocessor1610 to initiate a particular mode. Certain executingmode instructions1623 can initiate thecommunication sub-system1640 to establish one or more wireless communication links with the self-propelled device and/or the mobile computing device. Execution of acontrol mode1622 by theprocessor1610 may cause theprocessor1610 to generate distinct feedback responses using thefeedback mechanism1630 based on sensor data from the sensor(s)1660 indicating user gestures performed by the user.
• In some examples, thememory resources1620 can comprise a number ofstate machines1624 which can provide state machine reports1627 to theprocessor1610 can specified sensor patterns are identified byrespective states machines1624. Eachstate machine1624 may monitor for a single sensor pattern which, if identified by thatstate machine1624, can cause thestate machine1624 to transition states, thereby providing a state machine report1627 to theprocessor1610 identifying the user gesture performed. Theprocessor1610 can translate the state machine reports1627—which indicate the user gestures—in accordance with an executing set ofmode instructions1623 in order to generate a corresponding output via thefeedback mechanism1630 and/orcontrol commands1612 to be communicated to the self-propelled device via thecommunication sub-system1640.
• While examples ofFIG. 14,FIG. 15, andFIG. 16 provide for acomputer system1400, acomputing device1500, and aportable sensing device1600 for implementing aspects described, in some variations, other devices of the three can be arranged to implement some or all of the functionality described with the processing resources of the remotely operateddevice140 ofFIG. 1A, themobile computing device495 ofFIG. 4, or thewearable device102 ofFIG. 1A, as shown and described throughout.
• With further reference to examples ofFIG. 14,FIG. 15, andFIG. 16, some examples include functionality for projecting an orientation and/or perspective of a user onto a gaming environment via sensing output of theportable sensing device1600. For example, when theportable sensing device1600 is worn, the orientation and perspective of the user can be inferred from sensors1660 (e.g., IMU), and this sensor information can be virtualized for the gaming environment. For example, the gaming environment can be shown on a computing device (e.g., display screen of a computer, mobile computing device etc.). The gaming environment can a perspective that is based on the orientation of the user (e.g., user is standing north), as determined by theportable sensing device1600. The perspective can change as the user changes orientation, moves in a particular direction etc. In some examples, theportable sensing device1600 can be used to control a virtual or actual object (e.g., self-propelled device or remotely operated device), and the orientation and direction of the controlled object may be with reference to a reference frame of the user.
• In variations, a reference frame of the self-propelled device may be used, and the user's orientation can be used to influence control of the virtual or actual device in motion. For example, the user's movement or motion can influence a change of direction. Alternatively, both orientations can be used concurrently. For example, if the device under control is a virtual vehicle that carries the user, the user may turn his head (e.g., when wearing a necklace carrying the portable sensing device2000) to see a view to a particular side while the orientation of the vehicle is used for the motion of the vehicle.
• Multi-Device Usage
• FIG. 17 illustrates an embodiment of multiple sensing devices that concurrently provide input for a program or application which utilizes the inputs, along with inferences which can be made about a person or object that carries the devices, according to one or more examples. In particular, an example such as shown enables input from multiple sensing devices to be used for purpose of enabling inferences of movement and pose from two relevant sources of user motion. For example, inFIG. 17, auser1701 carries wearable devices in the form of awrist device1710 and pendent1712. In other examples, one or both of thewrist device1710 and pendent1712 can be in the form of an alternative form factor or device type. For example, the combination of sensing devices can include a hat, a ring, eyeglasses or a device which the user can carry in his or her hand (e.g., FOB, mobile computing device). In variations, more than two wearable devices can be employed by one user.
• FIG. 18 illustrates a system which concurrently utilizes input from multiple modular sensing devices in connection with execution of an application or program. With reference to an example ofFIG. 18, a multi-device system1800 includes a firstmodular sensing device1810, a second modular sensing device1820, and acontroller1830. Each of the first and secondmodular sensing devices1810,1820 includes a respective inertial measurement unit (IMU)1812,1822, aprocessor1814,1824 andmemory1816,1826. TheIMU1812,1822 of eachmodular sensing device1810,1820 can include sensors such as anaccelerometer1815,1825 andgyroscopic sensor1817,1827. The first and secondmodular sensing devices1810,1820 may also include additional sensing resources, such as a magnetometer and/or proximity sensor.
• Thecontroller1830 can include aprocessor1832 and amemory1834. Theprocessor1832 can execute instructions1835 for a program or application that can execute andprocess inputs1811,1813 from each of the respectivemodular sensing devices1810,1820. In some variations, thecontroller1830 is a mobile computing device, such as a multi-purpose wireless communication device which can wirelessly communicate with each of the first and secondmodular sensing devices1810,1820.
• While an example ofFIG. 18 illustrates thecontroller1830 as a separate device from the first and secondmodular sensing devices1810,1820, variations provide that thecontroller1830 is integrated or otherwise combined with at least one of the first or secondmodular sensing devices1810,1820. For example, thecontroller1830 can include a multi-purpose wireless communication device that is equipped with a gyroscopic sensor and accelerometer. Thus, for example, variations can provide the second modular sensing device1820 to be a local resource of thecontroller1830, which communicates with the firstmodular sensing device1810.
• With further reference toFIG. 18, thecontroller1830 can receiveinputs1811,1813 from respective first and secondmodular sensing devices1810,1820. Theinputs1811,1813 can be received in connection with anapplication1839 or program that is executed by theprocessor1832 of thecontroller1830. Theprocessor1832 can execute theinstructions1845 in order to implement an inference engine1835 for determining inferences about the person or object with one or both of themodular sensing devices1810,1820. For example, theapplication1839 can correspond to a game or simulation, and the inference engine1835 can be specific to theapplication1839. Among other applications, the inference engine1835 can be used to determine when the motions of twomodular sensing devices1810,1820 are separate and distinct from one another, or continuous and/or part of the same input motion.
• According to one implementation, eachinput1811,1813 can correspond to one or more of a position, height, orientation, velocity, linear and/or rotational acceleration. Each of the first andsecond sensing devices1810,1820 generate a set of measured (or sensed data) corresponding to, for example, a movement (e.g., gesture) made with therespective sensing device1810,1820. Additionally, thecontroller1830 can processinput1811,1813 corresponding to each of the respective data sets in order to determine a third data set of inferences. In this way, the inferences reflect information determined from sensed data, rather than directly measured data. The inferences which can be output from the inference engine1835 can be determinative or probabilistic, depending on implementation.
• With reference to an example ofFIG. 17,user1701 can wear two modular sensing devices, and the inference engine1835 can assume some inferences based on anatomical constraints and/or context (e.g., such as provided from execution of the application1839). For example, each of the first and secondmodular sensing devices1810,1820 can correspond to a wearable wrist device. Alternatively, the second modular sensing device1820 can correspond to the pendent1712 or neck-worn device. By way of example, if the first modular sensing device1810 (wrist device1710) is detected to be in motion, the inference engine1835 can be used to determine additional position data for the movement of that device along a third axis based on orientation, position or context of second modular sensing device1820 (wrist device1711 or pendent device1712). For example, if the first modular sensing device1810 (wrist device1711) measures arc motion, and the second modular sensing1820 is the pendent, then the orientation of the second modular sensing device can indicate whether, for example, the arc motion is in front of the user or to the user's side. Alternatively, if the second modular sensing device1820 is thesecond wrist device1712, the information sensed from the second wrist device can identify the corresponding hand or device as being in front of the body. In such an orientation, the inference engine1835 can determine the inference to be that the user is making the arc of motion in front of his body. Similarly, if the height of the second sensing device1820 is determined to be belt high and the device is held by the user, the orientation of the user's torso can be inferred (along with the direction of the arc).
• In examples in which the second modular sensing device1820 is a pocket device (e.g., mobile computing device, FOB), information can be determined from, for example, the height of the device (e.g., user standing, crouching or jumping) and the rotation of the device. For example, if the second modular sensing device1820 is pocket word, a change in the orientation of the device from vertical to horizontal, in combination with a downward acceleration can indicate the user is crouching. If the user is crouching, for example, the type of motion that is likely by the firstmodular sensing device1810 may be limited (e.g., motion of thewrist device1710 is likely in front of user when user is moving up or down). The examples described with respect toFIGS. 17 and 18 can enable the user to utilize the modular sensing device(s) in connection with a real-world gameplay environment (or other task-oriented activities) executed by one or more of themodular sensing devices1810,1820, control a remotely operated device using gestures sensed by themodular sensing devices1810,1820, interact with other users, and perform various tasks in which themodular sensing devices1810,1820 can provide feedback and response output.
• Modular Sensing Device Implementations
• FIG. 19 illustrates an example of a modular sensing device insertable into a wrist worn apparatus. In particular, amodular sensing device1900 can be constructed in accordance with examples provided herein, in order to implement operations and functionality such provided by any of the examples described. Themodular sensing device1900 can include ahousing1910 for containing a processor, memory (e.g., a controller implementing a plurality of state machines), and one or more sensors (e.g., IMU, gyroscope, accelerometer, proximity sensor, magnetometer, etc.). Themodular sensing device1900 can also include a wireless communication resource for communicating with other devices, including devices which may be controlled in movement (e.g., self-propelled device) or other processes.
• In some examples, themodular sensing device1900 can communicate sensor data, including output from the IMU, to another device for purpose of controlling movement of the other device. In some variations, themodular sensing device1900 can include processing capabilities to process raw sensor data into higher data forms of communication. For example, themodular sensing device1900 can generate output in the form of commands, or input for command selection from a receiving device.
• According to some examples, thehousing1910 of themodular sensing device1900 can include securement features for enabling themodular sensing device1900 to fasten onto anothercompatible structure1920. Thecompatible structure1920 can include an opening that is shaped to receive and secure themodular sensing device1910. As shown with an example ofFIG. 19, the securement features can include, for example, structural or shaped features of thehousing1910. For example, thehousing1910 can be dimensioned and/or structured (e.g., housing may be biased) to snap-fit into thecompatible structure1920. Alternatively, at least one of thehousing1910 orcompatible structure1920 can include an integrated and/or mechanical fastener to secure themodular sensing device1900.
• FIG. 20 illustrates an implementation of themodularized sensing device2000. As shown, thesensing device2000 can be retained by the compatible structure2020 (e.g., wrist-worn strap), and then removed and placed in an opening of a wielded device2010 (e.g., play sword). The placement of themodular sensing device2000 in differentcompatible structures2020,2010 for retention and use can be coordinated with different functionality being enabled through the sensing device. For example, themodular sensing device2000 in the wrist-wornstrap2020 can be used in conjunction with a first program running on a mobile computing device (controller), self-propelled device and/or other computer system (e.g., virtual gaming system). When placed in the wielded device2010 (e.g., a wand), themodular sensing device2000 can be operated in conjunction with a mobile computing device, self-propelled device and/or other computer system (e.g., virtual gaming system) which executes a second program or application. In each context, the orientation of themodular sensing device2000 can be used to determine a perspective, such as a virtual field of view for gameplay. The perspective can refer to the orientation, direction and/or position of the user, and/or of the user's body part with respect to the sensing device. With the wand, the orientation and direction of the sensing device can be used to project a corresponding virtual object in a virtual environment (e.g., sword). Themodular sensing device2000 may also be able to read an identifier of thecompatible structure2020,2010 in order to determine information about the structure, such as its dimension, and whether the structure is word or carried. Based on known information, inferences can be determined for purpose of virtualization, etc. (e.g., length of sword).
• Conclusion
• It is contemplated for examples described herein to extend to individual elements and concepts described herein, independently of other concepts, ideas or system, as well as for examples to include combinations of elements recited anywhere in this application. Although examples are described in detail herein with reference to the accompanying drawings, it is to be understood that this disclosure is not limited to those precise examples. As such, many modifications and variations will be apparent to practitioners skilled in this art. Accordingly, it is intended that the scope of this disclosure be defined by the following claims and their equivalents. Furthermore, it is contemplated that a particular feature described either individually or as part of an example can be combined with other individually described features, or parts of other examples, even if the other features and examples make no mentioned of the particular feature. Thus, the absence of describing combinations should not preclude the inventor from claiming rights to such combinations.

Claims (20)

What is claimed is:

1. A wearable device to be worn by a user, the wearable device comprising:

a mode selector;

one or more sensors;

a signal detector to detect wireless signals;

a feedback mechanism configured to provide feedback to the user, the feedback mechanism comprising at least one of a haptic system, an audio system, or a visual system; and

a controller configured to:

in response to a user input on the mode selector, initiate a selected mode from a plurality of modes on the wearable device, the selected mode causing the controller to execute a real-world game; and

in accordance with the selected mode, interpret sensor data from the one or more sensors and inputs from the signal detector, the sensor data corresponding to a series of actions performed by the user utilizing the wearable device; and

based on interpreting the sensor data and the detected signals in accordance with the selected mode, generate feedback responses via a feedback mechanism.

2. The wearable device ofclaim 1, wherein the wearable device comprises a wrist-worn device, and wherein the selected mode comprises a wield mode that causes the controller to:

detect sensor data corresponding to a series of actions performed by the user while the user is holding a wielded device; and

generate a series of feedback responses unique to the wield mode in response to the series of actions performed by the user.

3. The wearable device ofclaim 1, wherein the selected mode comprises a finder mode that causes the controller to:

using the signal detector, detect a wireless signal from a network device, the wireless signal including a unique identifier for the network device; and

determine whether the unique identifier for the network device correlates to any of a plurality of digital resources.

4. The wearable device ofclaim 3, wherein the finder mode further causes the controller to:

in response to determining that the unique identifier for the network device does correlate to a respective digital resource, (i) generate a feedback response via the feedback mechanism indicating that the network device comprises a source of the respective digital resource, and (ii) enable the user to select a mining mode on the wearable device to cause the controller to store an amount of the respective digital resource.

5. The wearable device ofclaim 4, wherein the mining mode causes the controller to:

generate variable feedback outputs via the feedback mechanism based on a direction towards the network device, based on the wireless signal, to enable the user to locate the respective digital resource.

6. The wearable device ofclaim 5, wherein the mining mode causes the controller to generate the variable feedback outputs by (i) dynamically increasing an intensity of the feedback mechanism as the user points the wearable device towards the direction of the network device, and (ii) dynamically decreasing an intensity of the feedback mechanism as the user points the wearable device away from the direction of the network device.

7. The wearable device ofclaim 5, wherein the mining mode further causes the controller to:

when the user points the wearable device in the direction of the network device for a predetermined amount of time, generate and store a predetermined amount of the respective digital resource in a memory of the wearable device.

8. The wearable device ofclaim 7, wherein the selected mode enables the user to utilize the respective digital resource in connection with the real-world game.

9. The wearable device ofclaim 7, wherein the selected mode further causes the controller to:

wirelessly connect with a computing device of the user; and

transfer the respective digital resource to a memory of the computing device, the respective digital resource being utilizable in connection with gameplay on the computing device.

10. The wearable device ofclaim 1, wherein the selected mode further causes the controller to:

while executing the real-world game, detect a proximate wearable device using the signal detector;

wirelessly connect with the proximate wearable device via a communication interface of the wearable device; and

in response to wirelessly connecting with the proximate wearable device, execute a sub-mode to cause the controller generate feedback responses based on user interactions between the user and a user of the proximate wearable device.

11. The wearable device ofclaim 10, wherein the selected mode further causes the controller to:

detect, via an inductive interface of the wearable device, an inductive pairing between the wearable device and the proximate wearable device;

wherein the controller wirelessly connects with the proximate wearable device and executes the sub-mode in response to the inductive pairing.

12. The wearable device ofclaim 11, wherein the sub-mode comprises an interactive mode that enables the user and a user of the proximate wearable device to initiate a simulated fighting session that causes the controller to reconfigure interpretation of the sensor data to generate the feedback responses based on the user interactions.

13. The wearable device ofclaim 12, wherein the interactive mode further causes the controller to:

alternate between attacking and blocking sub-modes between the user and the proximate user.

14. The wearable device ofclaim 13, wherein the attacking sub-mode causes the controller to:

identify, in the sensor data, a set of actions in the series of actions performed by the user that indicates an attacking move;

translate the set of actions to offensive data corresponding to the attack move; and

transmit the offensive data to the proximate wearable device via the wireless communication interface.

15. The wearable device ofclaim 14, wherein the attacking sub-mode further causes the controller to:

receive, via the wireless communication interface, an indication of whether the attack move was successful; and

generate a feedback output using the feedback mechanism based on the indication.

16. The wearable device ofclaim 15, wherein the blocking sub-mode causes the controller to:

identify, in the sensor data, a second set of actions performed by the user indicating a blocking move;

translate the second set of actions into defensive data corresponding to the blocking move; and

transmit the defensive data to the proximate device via the wireless communication interface, wherein the defensive data indicates whether the blocking move was successful.

17. The wearable device ofclaim 16, wherein the interactive mode further causes the controller to:

generate a respective score for each successful attacking move and each successful blocking move;

maintain a score count based on the respective scores for each of the successful attacking moves and each of the successful blocking moves; and

when the score count reaches a predetermined threshold, generate a feedback response using the feedback mechanism indicating that the simulated fighting session has ended.

18. The wearable device ofclaim 17, wherein the interactive mode further causes the controller to:

connect with a mobile computing device; and

transmit a final score count to the mobile computing device for display.

19. The wearable device ofclaim 1, wherein the controller implements a plurality of state machines, each respective state machine being associated with a sensor data signature corresponding to a specified user action in the series of actions, and wherein the respective state machine executes a state transition when the sensor data correlates with the sensor data signature, the state transition causing the controller to generate an associated feedback response via the feedback mechanism.

20. A computer-implemented method of operating a wearable device, the method being performed by one or more processors and comprising:

in response to a user input on a mode selector of the wearable device, initiating a selected mode from a plurality of modes on the wearable device, the selected mode causing the wearable device to execute a real-world game; and

in accordance with the selected mode, interpreting sensor data from one or more sensors of the wearable device and detected signals from a signal detector of the wearable device, the sensor data corresponding to a series of actions performed by the user utilizing the wearable device; and

based on interpreting the sensor data and the detected signals in accordance with the selected mode, generating feedback responses via a feedback mechanism.

Images